Compare commits

...
Author SHA1 Message Date
EthanHealy01 3869072cf7 Merge branch 'main' of https://github.com/Stirling-Tools/Stirling-PDF into feature/documentClassifierAgent 2026-07-02 15:33:53 +01:00
EthanHealy01 3817779b9e translation 2026-07-02 14:42:07 +01:00
EthanHealy01 c4e66f2c2d Merge branch 'feature/documentClassifierAgent' of https://github.com/Stirling-Tools/Stirling-PDF into feature/documentClassifierAgent 2026-07-02 14:28:07 +01:00
EthanHealy01 c27fd4db69 added the classification taxonomy table in the db, team scoped. Allow admins to edit the taxonomy and set up the backend to query it and send it to python each time it's requested. No longer requires a backend restart to make it get the latest taxonomy. Also fixed up how the ordeering of policies is selected and how they chain together in that order. Also made a tiny fix to the my files section to keep it up to date with the indexdb 2026-07-02 14:28:02 +01:00
Anthony StirlingandGitHub 6c85200eb9 Add portal access control and S3/MCP/API integration configs (#6795) 2026-07-01 13:49:02 +01:00
Reece BrowneandGitHub 467f3a86c4 Portal policies (#6852) 2026-07-01 13:42:26 +01:00
Anthony StirlingandGitHub 9d3701a585 Fix rearrange-pages DUPLICATE producing shared page nodes (pypdf cyclic-references CI break) (#6851) 2026-07-01 13:40:27 +01:00
James BruntonandGitHub c22ecc6c09 Add counts to sources page (#6819) 2026-07-01 11:42:35 +01:00
Reece BrowneGitHubaikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>Connor Yoh
b38c849726 Portal: Procurement surface — layout rework + stateful mock backend (#6785)
Co-authored-by: aikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>
Co-authored-by: Connor Yoh <con.yoh13@gmail.com>
2026-07-01 11:38:38 +01:00
EthanHealy01 0ae7052dcb Merge branch 'main' of https://github.com/Stirling-Tools/Stirling-PDF into feature/documentClassifierAgent 2026-07-01 10:55:36 +01:00
41f1cb2c22 build(deps): bump test pypdf + add translations (#6831)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-30 23:40:26 +01:00
Anthony StirlingandGitHub ff3e3bd0fc Add desktop hardware token signing and trust-aware signature validation (#6765)
# Description of Changes

<img width="432" height="800" alt="image"
src="https://github.com/user-attachments/assets/a01ed9ac-220c-4911-9134-b51e0f321be8"
/>

<img width="408" height="859" alt="image"
src="https://github.com/user-attachments/assets/a9c285b6-5b75-493a-95ec-09e08d0f58f1"
/>

<img width="426" height="874" alt="image"
src="https://github.com/user-attachments/assets/a60db96e-be93-4cc5-ba0a-63512c2857ba"
/>

<img width="356" height="1076" alt="image"
src="https://github.com/user-attachments/assets/24d03674-94d3-40ed-99ee-73395bafae6a"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-30 23:23:58 +01:00
EthanHealy01andGitHub 54042c8e5e Signing UI edge-case cleanup (#6849) 2026-06-30 23:04:10 +01:00
bb92ecc143 Update Backend 3rd Party Licenses + Translations and bump versio (#6794)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
2026-06-30 22:41:40 +01:00
EthanHealy01andGitHub 7ab30d2629 add file share to the top workbench bar and add shared signing (#6715) 2026-06-30 22:14:49 +01:00
EthanHealy01andGitHub 69fc4d5bc1 Merge branch 'main' into feature/documentClassifierAgent 2026-06-30 19:42:24 +01:00
James BruntonandGitHub 276eb8f2a7 Add pipelines page to portal (#6818)
# Description of Changes

Connect pipelines page to the backend. Note that this is really half an
implementation because the portal doesn't have access to the tools list
and their settings, but I can't fix that without re-architecture work,
which I'll do in another PR, then come back to finish this off in a new
PR.

<img width="786" height="579" alt="image"
src="https://github.com/user-attachments/assets/d3f06110-a35d-4d48-a2f9-1edb900c5c35"
/>

<img width="1232" height="519" alt="image"
src="https://github.com/user-attachments/assets/9f344648-ea45-498d-9e84-9558a3999838"
/>
2026-06-30 16:11:48 +00:00
James BruntonandGitHub e44da5c410 Fix missing refresh token on desktop (#6838)
# Description of Changes
Fix #6801, along with fixing policies on desktop, which would attempt to
download policy outputs from the local backend instead of the server,
where they actually live. I've changed the policies logic to maintain
the same backend for the file retrieval as it used for the policy
running, so when we support running policies locally, it should still
work correctly.
2026-06-30 14:07:12 +00:00
Reece BrowneandGitHub 0beff1a92b feat(shared): make @shared the single home for brand logo assets (#6714)
## What

Makes `@shared` the single home for the Stirling brand logo assets.
Moves the editor's two logo sets — `classic-logo` + `modern-logo` (22
files: marks, wordmarks, favicons, login headers, PNGs) — out of
`editor/public/` into `shared/assets/brand/`, and adds a Storybook
**Brand/Logos** gallery.

## Why this shape (not a plain move)

The editor serves logos by **URL** from `public/` and switches
`classic`/`modern` by a **user preference** (`useLogoAssets`,
`manifest.json` / `manifest-classic.json`, `index.html` favicon links).
Rewiring all that to module imports would be a large, risky change to
the variant system.

Instead the editor keeps its variant system **unchanged** and just
sources the files from shared: `vite-plugin-static-copy` copies
`shared/assets/brand/{classic,modern}-logo/*` back to the served
`/{classic,modern}-logo` paths (the editor already uses this plugin for
pdfium/pdfjs assets). Single source of truth in shared, zero editor
code/manifest/markup changes.

## Verified

- **Build:** editor builds with both sets present at
`dist/{modern,classic}-logo/`; `manifest.json` + favicon refs resolve.
- **Dev:** the vite dev server serves the bridged paths —
`/modern-logo/logo512.png`,
`/modern-logo/StirlingPDFLogoNoTextDark.svg`,
`/classic-logo/favicon.ico` all return **HTTP 200** (the plugin's dev
middleware).
- Typecheck clean on core/proprietary/saas; prettier clean; `storybook
build` succeeds with the `Brand/Logos` gallery bundled.
- The portal's existing `@shared/assets` brand imports are untouched.

## Follow-ups (not in this PR)

- **Dedup:** `shared/assets/stirling-mark-*.svg` is byte-identical to
`brand/modern-logo/StirlingPDFLogoNoTextDark.svg`, and
`stirling-pdf-logo-*` is a near-twin of the modern wordmark. Reconciling
these (and re-pointing the portal) needs a designer eye on which
wordmark is canonical, so it's left out here to avoid changing the
portal's rendered logo.
- `editor/src/logo.svg` appears unused (no references) — candidate for
deletion separately.
2026-06-30 11:24:14 +00:00
EthanHealy01andGitHub 2e023a6e78 Merge branch 'main' into feature/documentClassifierAgent 2026-06-30 10:45:27 +01:00
EthanHealy01 7e523d48d7 rename script and remove retry test 2026-06-29 22:27:26 +01:00
EthanHealy01andGitHub b8cb020e59 Merge branch 'main' into feature/documentClassifierAgent 2026-06-29 22:10:52 +01:00
EthanHealy01 a92722ff13 remove classify as a user facing tool 2026-06-29 22:10:09 +01:00
EthanHealy01 fed7ad300f document classifier agent that runs as a policy and assigns a category and sub-category docType 2026-06-29 21:58:03 +01:00
433 changed files with 29010 additions and 16037 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.13.2
pkgver=2.14.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.13.2
pkgver=2.14.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
+8
View File
@@ -19,3 +19,11 @@ frontend/shared/components/CodeBlock.stories.tsx:curl-auth-header:4
# Truncated placeholder API key in portal docs example (sk_live_8f2c...e10) - not a real secret.
frontend/portal/src/components/docs/GettingStartedSection.tsx:generic-api-key:31
# False positive: generic-api-key matches the Java type name "X509Certificate"
# in a method signature (CreateSignatureBase.resolveSignatureAlgorithm) - not a secret.
app/core/src/main/java/org/apache/pdfbox/examples/signature/CreateSignatureBase.java:generic-api-key:224
# Supabase publishable key (public by design, RLS-protected) used as a CI fallback
# default in the tauri-build workflow when the GitHub secret is unset - not a real secret.
.github/workflows/tauri-build.yml:generic-api-key:402
+26 -10
View File
@@ -149,16 +149,32 @@ tasks:
# Pin jlink to JAVA_HOME so the bundled JRE matches the JDK the build
# uses. Bare `jlink` on PATH can resolve to an older system Java (the
# ubuntu runner ships Java 11), producing a runtime jlink:verify rejects.
- |
JLINK="${JAVA_HOME:+$JAVA_HOME/bin/}jlink"
JLINK_COMPRESS="$("$JLINK" --help 2>&1 | grep -q 'zip-\[0-9\]' && echo zip-6 || echo 2)"
"$JLINK" \
--add-modules {{.JLINK_MODULES}} \
--strip-debug \
--compress="$JLINK_COMPRESS" \
--no-header-files \
--no-man-pages \
--output runtime/jre
#
# jdk.crypto.mscapi (the Windows certificate store / SunMSCAPI provider, used by
# hardware-backed cert signing) is a Windows-only module - it only exists in a Windows
# JDK's jmods, so it is added on Windows only or jlink fails to resolve it elsewhere.
- cmd: |
JLINK="${JAVA_HOME:+$JAVA_HOME/bin/}jlink"
JLINK_COMPRESS="$("$JLINK" --help 2>&1 | grep -q 'zip-\[0-9\]' && echo zip-6 || echo 2)"
"$JLINK" \
--add-modules {{.JLINK_MODULES}},jdk.crypto.mscapi \
--strip-debug \
--compress="$JLINK_COMPRESS" \
--no-header-files \
--no-man-pages \
--output runtime/jre
platforms: [windows]
- cmd: |
JLINK="${JAVA_HOME:+$JAVA_HOME/bin/}jlink"
JLINK_COMPRESS="$("$JLINK" --help 2>&1 | grep -q 'zip-\[0-9\]' && echo zip-6 || echo 2)"
"$JLINK" \
--add-modules {{.JLINK_MODULES}} \
--strip-debug \
--compress="$JLINK_COMPRESS" \
--no-header-files \
--no-man-pages \
--output runtime/jre
platforms: [linux, darwin]
# jlink emits its files mode 444 (read-only). Tauri's build-script
# resource copier preserves source permissions when staging
# `runtime/jre/**/*` into `target/<profile>/runtime/jre/...`, so the
+21
View File
@@ -55,6 +55,15 @@ tasks:
- editor/src/core/data/ogImageMap.json
- editor/public/og-metadata.json
prepare:classifier-categories:
internal: true
run: when_changed
desc: "Regenerate the engine classifier categories JSON from the TS source of truth"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts
sources:
- editor/src/proprietary/data/classificationTaxonomy.ts
prepare:
desc: "Set up dev environment"
run: when_changed
@@ -65,6 +74,7 @@ tasks:
vars: { MODE: '{{.MODE}}' }
- prepare:icons
- prepare:og
- prepare:classifier-categories
# ============================================================
# Development
@@ -401,12 +411,23 @@ tasks:
cmds:
- node editor/scripts/generate-og-metadata.mjs --check
classifier-categories:
desc: "Regenerate the engine classifier categories JSON from the TS source"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts
classifier-categories:check:
desc: "Fail if the committed classifier categories JSON is out of date"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts --check
check:all:
desc: "Full CI quality gate"
cmds:
# Runs first, before prepare regenerates: guards the committed og-metadata.json /
# ogImageMap.json that the Cloudflare Pages (plain `vite build`) deploy relies on.
- task: og:check
- task: classifier-categories:check
- task: typecheck:all
- task: lint
- task: format:check
+27
View File
@@ -95,6 +95,7 @@ tasks:
vars:
PORT: '{{.PORTAL_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
MOCKS: 'false'
OPEN: "true"
dev:portal:all:
@@ -117,6 +118,32 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
# Point the portal's "Editor" app switcher at the editor we spawn here.
EDITOR_URL: 'http://localhost:{{.EDITOR_PORT}}/'
MOCKS: 'false'
OPEN: "true"
- task: frontend:dev
vars:
PORT: '{{.EDITOR_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
dev:portal:all:saas:
desc: "Start SaaS backend + developer portal + editor concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5174{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5174{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
PORTAL_PORT: '{{index (splitList "\n" .PORTS) 1}}'
EDITOR_PORT: '{{index (splitList "\n" .PORTS) 2}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
POLICIES_ENABLED: "true"
- task: frontend:dev:portal
vars:
PORT: '{{.PORTAL_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
EDITOR_URL: 'http://localhost:{{.EDITOR_PORT}}/'
MOCKS: 'false'
OPEN: "true"
- task: frontend:dev
vars:
@@ -7,6 +7,7 @@ import java.time.format.DateTimeFormatter;
import java.util.Calendar;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
@@ -17,6 +18,11 @@ import stirling.software.common.model.PdfMetadata;
@Service
public class PdfMetadataService {
/**
* ({@code {category, docType, typeConfidence, tags}}). Written by the classify-and-tag tool.
*/
public static final String CLASSIFICATION_KEY = "StirlingPDFClassification";
private final ApplicationProperties applicationProperties;
private final String stirlingPDFLabel;
private final UserServiceInterface userService;
@@ -177,4 +183,14 @@ public class PdfMetadataService {
}
pdf.getDocumentInformation().setAuthor(author);
}
/**
* Write the document classifier's JSON result into the custom Info-dictionary field {@link
* #CLASSIFICATION_KEY}, leaving all other metadata untouched.
*/
public void setClassificationMetadata(PDDocument pdf, String classificationJson) {
PDDocumentInformation info = pdf.getDocumentInformation();
info.setCustomMetadataValue(CLASSIFICATION_KEY, classificationJson);
pdf.setDocumentInformation(info);
}
}
@@ -24,12 +24,14 @@ import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.PrivateKey;
import java.security.Provider;
import java.security.UnrecoverableKeyException;
import java.security.cert.Certificate;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.Locale;
import org.apache.pdfbox.pdmodel.interactive.digitalsignature.SignatureInterface;
import org.bouncycastle.cert.jcajce.JcaCertStore;
@@ -50,6 +52,13 @@ public abstract class CreateSignatureBase implements SignatureInterface {
@Getter private Certificate[] certificateChain;
@Setter private String tsaUrl;
/**
* Provider that must service the signing operation. Set for hardware-held keys (SunPKCS11 for
* USB tokens, SunMSCAPI for the Windows store) so the {@link java.security.Signature} runs on
* the token. Left {@code null} for software keystores, which use the default provider.
*/
@Setter private Provider signingProvider;
/**
* Specifies whether the external signing scenario should be used. If set to {@code true},
* external signing will be performed and {@link SignatureInterface} will be used for signing.
@@ -80,25 +89,48 @@ public abstract class CreateSignatureBase implements SignatureInterface {
NoSuchAlgorithmException,
IOException,
CertificateException {
// grabs the first alias from the keystore and get the private key. An
// alternative method or constructor could be used for setting a specific
// alias that should be used.
this(keystore, pin, null);
}
/**
* Initialize the signature creator, optionally selecting a specific certificate by alias. A
* hardware token / the Windows store can hold several certificates, so the caller picks one;
* when {@code requestedAlias} is null the first usable entry is used (software keystore
* behaviour).
*
* @param keystore the keystore (software, PKCS#11 or Windows-MY)
* @param pin the keystore / token PIN, may be null for the Windows store
* @param requestedAlias the alias to sign with, or null to pick the first usable entry
*/
public CreateSignatureBase(KeyStore keystore, char[] pin, String requestedAlias)
throws KeyStoreException,
UnrecoverableKeyException,
NoSuchAlgorithmException,
IOException,
CertificateException {
if (requestedAlias != null
&& !requestedAlias.isBlank()
&& keystore.containsAlias(requestedAlias)) {
privateKey = (PrivateKey) keystore.getKey(requestedAlias, pin);
certificateChain = resolveChain(keystore, requestedAlias);
if (certificateChain == null) {
throw new IOException("Could not find certificate for alias " + requestedAlias);
}
checkValidity(certificateChain[0]);
return;
}
// grabs the first alias from the keystore and gets the private key.
Enumeration<String> aliases = keystore.aliases();
String alias;
Certificate cert = null;
while (cert == null && aliases.hasMoreElements()) {
alias = aliases.nextElement();
String alias = aliases.nextElement();
privateKey = (PrivateKey) keystore.getKey(alias, pin);
Certificate[] certChain = keystore.getCertificateChain(alias);
Certificate[] certChain = resolveChain(keystore, alias);
if (certChain != null) {
certificateChain = certChain;
cert = certChain[0];
if (cert instanceof X509Certificate) {
// avoid expired certificate
((X509Certificate) cert).checkValidity();
//// SigUtils.checkCertificateUsage((X509Certificate) cert);
}
checkValidity(cert);
}
}
@@ -107,6 +139,27 @@ public abstract class CreateSignatureBase implements SignatureInterface {
}
}
/**
* Resolve the certificate chain for an alias. PKCS#11 tokens and the Windows store frequently
* expose only the leaf certificate (a null chain), so fall back to the single certificate.
*/
private static Certificate[] resolveChain(KeyStore keystore, String alias)
throws KeyStoreException {
Certificate[] chain = keystore.getCertificateChain(alias);
if (chain != null && chain.length > 0) {
return chain;
}
Certificate single = keystore.getCertificate(alias);
return single != null ? new Certificate[] {single} : null;
}
private static void checkValidity(Certificate cert) throws CertificateException {
if (cert instanceof X509Certificate x509Cert) {
// avoid expired certificate
x509Cert.checkValidity();
}
}
public final void setPrivateKey(PrivateKey privateKey) {
this.privateKey = privateKey;
}
@@ -136,12 +189,18 @@ public abstract class CreateSignatureBase implements SignatureInterface {
try {
CMSSignedDataGenerator gen = new CMSSignedDataGenerator();
X509Certificate cert = (X509Certificate) certificateChain[0];
ContentSigner sha1Signer =
new JcaContentSignerBuilder("SHA256WithRSA").build(privateKey);
JcaContentSignerBuilder signerBuilder =
new JcaContentSignerBuilder(resolveSignatureAlgorithm(privateKey, cert));
// Hardware keys (PKCS#11 / Windows store) must sign on their own provider so the
// operation runs on the token; software keys use the default provider.
if (signingProvider != null) {
signerBuilder.setProvider(signingProvider);
}
ContentSigner signer = signerBuilder.build(privateKey);
gen.addSignerInfoGenerator(
new JcaSignerInfoGeneratorBuilder(
new JcaDigestCalculatorProviderBuilder().build())
.build(sha1Signer, cert));
.build(signer, cert));
gen.addCertificates(new JcaCertStore(Arrays.asList(certificateChain)));
CMSProcessableInputStream msg = new CMSProcessableInputStream(content);
CMSSignedData signedData = gen.generate(msg, false);
@@ -157,4 +216,26 @@ public abstract class CreateSignatureBase implements SignatureInterface {
throw new IOException(e);
}
}
/**
* Pick a SHA-256 signature algorithm that matches the key type. RSA keeps the historical
* default; EC / EdDSA tokens are common, so they are handled too.
*/
private static String resolveSignatureAlgorithm(PrivateKey key, X509Certificate cert) {
String alg = key.getAlgorithm();
if (alg == null || alg.isBlank()) {
alg = cert.getPublicKey().getAlgorithm();
}
alg = alg == null ? "" : alg.toUpperCase(Locale.ROOT);
if (alg.contains("ED25519") || alg.contains("EDDSA")) {
return "Ed25519";
}
if (alg.contains("EC")) { // EC, ECDSA
return "SHA256withECDSA";
}
if (alg.contains("DSA")) {
return "SHA256withDSA";
}
return "SHA256withRSA";
}
}
@@ -3,9 +3,12 @@ package stirling.software.SPDF.controller.api;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import org.apache.pdfbox.cos.COSDictionary;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageTree;
@@ -261,10 +264,19 @@ public class RearrangePagesPDFController {
log.info("newPageOrder = {}", newPageOrder);
log.info("totalPages = {}", totalPages);
// Snapshot the desired pages before mutating the source document's page tree.
// Snapshot desired pages before mutating the tree; clone repeats (e.g. DUPLICATE)
// so each slot is a distinct node, not one PDPage under multiple /Kids.
List<PDPage> newPages = new ArrayList<>(newPageOrder.size());
Set<Integer> seenIndices = new HashSet<>();
for (Integer idx : newPageOrder) {
newPages.add(document.getPage(idx));
PDPage page = document.getPage(idx);
if (!seenIndices.add(idx)) {
// Duplicate index: distinct page node sharing content/resources.
COSDictionary clonedDict = new COSDictionary();
clonedDict.addAll(page.getCOSObject());
page = new PDPage(clonedDict);
}
newPages.add(page);
}
// Rearrange in-place on the source document rather than copying pages into a
@@ -350,6 +350,18 @@ public class ConfigController {
"serverCertificateEnabled",
serverCertificateService != null && serverCertificateService.isEnabled());
// Hardware-backed signing (Windows store / USB PKCS#11 tokens) is only viable on the
// desktop bundle, where the backend runs locally in the user's session. The Tauri
// bundle signals this via STIRLING_PDF_TAURI_MODE (machineType is Server-jar there);
// the bare-jar desktop launcher signals it via a Client-* machineType.
boolean hardwareSigningAvailable =
Boolean.parseBoolean(System.getProperty("STIRLING_PDF_TAURI_MODE", "false"));
if (!hardwareSigningAvailable && applicationContext.containsBean("machineType")) {
String mt = applicationContext.getBean("machineType", String.class);
hardwareSigningAvailable = mt != null && mt.startsWith("Client-");
}
configData.put("hardwareSigningAvailable", hardwareSigningAvailable);
// Legal settings
configData.put(
"termsAndConditions", applicationProperties.getLegal().getTermsAndConditions());
@@ -70,10 +70,13 @@ import io.micrometer.common.util.StringUtils;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.SPDF.model.api.security.SignPDFWithCertRequest;
import stirling.software.SPDF.service.HardwareKeyStoreService;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
@@ -109,14 +112,17 @@ public class CertSignController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ServerCertificateServiceInterface serverCertificateService;
private final TempFileManager tempFileManager;
private final HardwareKeyStoreService hardwareKeyStoreService;
public CertSignController(
CustomPDFDocumentFactory pdfDocumentFactory,
@Autowired(required = false) ServerCertificateServiceInterface serverCertificateService,
TempFileManager tempFileManager) {
TempFileManager tempFileManager,
HardwareKeyStoreService hardwareKeyStoreService) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.serverCertificateService = serverCertificateService;
this.tempFileManager = tempFileManager;
this.hardwareKeyStoreService = hardwareKeyStoreService;
}
public static void sign(
@@ -170,7 +176,8 @@ public class CertSignController {
"This endpoint accepts a PDF file, a digital certificate and related"
+ " information to sign the PDF. It then returns the digitally signed PDF"
+ " file. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<Resource> signPDFWithCert(@ModelAttribute SignPDFWithCertRequest request)
public ResponseEntity<Resource> signPDFWithCert(
@ModelAttribute SignPDFWithCertRequest request, HttpServletRequest httpRequest)
throws Exception {
MultipartFile pdf = request.getFileInput();
String certType = request.getCertType();
@@ -196,6 +203,8 @@ public class CertSignController {
KeyStore ks = null;
String keystorePassword = password;
Provider signingProvider = null;
HardwareKeyStoreService.Pkcs11Session pkcs11Session = null;
switch (certType) {
case "PEM":
@@ -245,6 +254,31 @@ public class CertSignController {
ks = serverCertificateService.getServerKeyStore();
keystorePassword = serverCertificateService.getServerCertificatePassword();
break;
case "WINDOWS_STORE":
hardwareKeyStoreService.assertLocalDesktop(httpRequest);
ks = hardwareKeyStoreService.loadWindowsKeyStore();
signingProvider = hardwareKeyStoreService.windowsProvider();
// PIN is prompted by the Windows CSP / token middleware, not passed here.
keystorePassword = password;
break;
case "PKCS11":
hardwareKeyStoreService.assertLocalDesktop(httpRequest);
char[] pkcs11Pin = password != null ? password.toCharArray() : null;
try {
pkcs11Session =
hardwareKeyStoreService.openPkcs11(
request.getPkcs11LibraryPath(),
request.getPkcs11Slot(),
pkcs11Pin);
} finally {
if (pkcs11Pin != null) {
java.util.Arrays.fill(pkcs11Pin, '\0');
}
}
ks = pkcs11Session.keyStore();
signingProvider = pkcs11Session.provider();
keystorePassword = password;
break;
default:
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
@@ -252,7 +286,9 @@ public class CertSignController {
"certificate type: " + certType);
}
CreateSignature createSignature = new CreateSignature(ks, keystorePassword.toCharArray());
char[] pin = keystorePassword != null ? keystorePassword.toCharArray() : null;
CreateSignature createSignature =
new CreateSignature(ks, pin, request.getAlias(), signingProvider);
TempFile signedOut = tempFileManager.createManagedTempFile(".pdf");
try (OutputStream os = new FileOutputStream(signedOut.getFile())) {
sign(
@@ -269,6 +305,14 @@ public class CertSignController {
} catch (IOException e) {
signedOut.close();
throw e;
} finally {
// Clear the PIN copy and log out the token session once signing is done.
if (pin != null) {
java.util.Arrays.fill(pin, '\0');
}
if (pkcs11Session != null) {
pkcs11Session.close();
}
}
// Return the signed PDF
return WebResponseUtils.pdfFileToWebResponse(
@@ -324,7 +368,22 @@ public class CertSignController {
NoSuchAlgorithmException,
IOException,
CertificateException {
super(keystore, pin);
this(keystore, pin, null, null);
}
public CreateSignature(
KeyStore keystore, char[] pin, String alias, Provider signingProvider)
throws KeyStoreException,
UnrecoverableKeyException,
NoSuchAlgorithmException,
IOException,
CertificateException {
super(keystore, pin, alias);
setSigningProvider(signingProvider);
loadLogo();
}
private void loadLogo() throws IOException {
ClassPathResource resource = new ClassPathResource("static/images/signature.png");
try (InputStream is = resource.getInputStream()) {
logoFile = Files.createTempFile("signature", ".png").toFile();
@@ -305,6 +305,23 @@ public class GetInfoOnPDF {
}
}
/**
* Info-dictionary keys exposed above via typed getters; any other key in the dictionary is
* surfaced as custom metadata (e.g. the classification policy's StirlingPDFClassification
* entry).
*/
private static final java.util.Set<String> STANDARD_INFO_KEYS =
java.util.Set.of(
"Title",
"Author",
"Subject",
"Keywords",
"Producer",
"Creator",
"CreationDate",
"ModDate",
"Trapped");
private static ObjectNode extractMetadata(PDDocument document) {
ObjectNode metadata = objectMapper.createObjectNode();
@@ -335,6 +352,18 @@ public class GetInfoOnPDF {
if (modificationDate != null) {
metadata.put("ModificationDate", modificationDate);
}
// Surface custom Info-dictionary entries (anything beyond the
// standard fields above) — e.g. StirlingPDFClassification
for (String key : info.getMetadataKeys()) {
if (STANDARD_INFO_KEYS.contains(key)) {
continue;
}
String value = info.getCustomMetadataValue(key);
if (value != null && !value.isBlank()) {
metadata.put(key, value);
}
}
}
} catch (Exception e) {
log.error("Error extracting metadata: {}", e.getMessage());
@@ -0,0 +1,85 @@
package stirling.software.SPDF.controller.api.security;
import java.util.List;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.security.HardwareCertificateInfo;
import stirling.software.SPDF.model.api.security.HardwareSigningCapabilities;
import stirling.software.SPDF.model.api.security.Pkcs11CertificatesRequest;
import stirling.software.SPDF.service.HardwareKeyStoreService;
/**
* Lets the desktop frontend discover which hardware-backed signing options the local backend can
* reach (Windows certificate store, plugged-in USB / PKCS#11 tokens) and enumerate the certificates
* available to sign with. Enumeration endpoints are restricted to the desktop bundle, reached over
* loopback - see {@link HardwareKeyStoreService#assertLocalDesktop}.
*/
@RestController
@RequestMapping("/api/v1/security/cert-sign/hardware")
@RequiredArgsConstructor
@Slf4j
@Tag(name = "Security", description = "Security APIs")
public class HardwareSigningController {
private final HardwareKeyStoreService hardwareKeyStoreService;
@GetMapping("/capabilities")
@Operation(
summary = "Hardware signing capabilities",
description =
"Reports whether hardware-backed signing is available on this device and which"
+ " PKCS#11 driver libraries were detected. Returns desktop=false when"
+ " not running as the desktop app.")
public ResponseEntity<HardwareSigningCapabilities> getCapabilities() {
return ResponseEntity.ok(hardwareKeyStoreService.capabilities());
}
@GetMapping("/windows-certificates")
@Operation(
summary = "List Windows certificate store signing certificates",
description =
"Enumerates certificates with a usable private key from the current user's"
+ " Windows certificate store. Desktop-only, loopback-only.")
public ResponseEntity<List<HardwareCertificateInfo>> getWindowsCertificates(
HttpServletRequest request) throws Exception {
hardwareKeyStoreService.assertLocalDesktop(request);
return ResponseEntity.ok(hardwareKeyStoreService.listWindowsCertificates());
}
@PostMapping("/pkcs11-certificates")
@Operation(
summary = "List PKCS#11 token signing certificates",
description =
"Logs into a PKCS#11 token with the supplied PIN and enumerates its signing"
+ " certificates. The PIN is used only for this call. Desktop-only,"
+ " loopback-only.")
public ResponseEntity<List<HardwareCertificateInfo>> getPkcs11Certificates(
HttpServletRequest request, @RequestBody Pkcs11CertificatesRequest body)
throws Exception {
hardwareKeyStoreService.assertLocalDesktop(request);
char[] pin = body.pin() != null ? body.pin().toCharArray() : null;
try {
return ResponseEntity.ok(
hardwareKeyStoreService.listPkcs11Certificates(
body.libraryPath(), body.slot(), pin));
} finally {
if (pin != null) {
java.util.Arrays.fill(pin, '\0');
}
}
}
}
@@ -102,8 +102,27 @@ public class ValidateSignatureController {
try (PDDocument document = pdfDocumentFactory.load(file.getInputStream())) {
List<PDSignature> signatures = document.getSignatureDictionaries();
// Detect content appended outside every signature's ByteRange (added after signing). A
// properly signed document has its last signature cover all the way to EOF; if the
// furthest any signature reaches stops short of the file length, the tail is unsigned.
// Taking the max across all signatures avoids false positives on legitimately
// multi-signed PDFs, where an earlier signature intentionally omits later revisions.
long fileLength = file.getSize();
long maxCovered = 0;
for (PDSignature sig : signatures) {
int[] byteRange = sig.getByteRange();
if (byteRange != null && byteRange.length == 4) {
long end = (long) byteRange[2] + byteRange[3];
if (end > maxCovered) {
maxCovered = end;
}
}
}
boolean documentCovered = maxCovered <= 0 || maxCovered >= fileLength;
for (PDSignature sig : signatures) {
SignatureValidationResult result = new SignatureValidationResult();
result.setCoversEntireDocument(documentCovered);
try {
byte[] signedContent = sig.getSignedContent(file.getInputStream());
@@ -0,0 +1,21 @@
package stirling.software.SPDF.model.api.security;
/**
* Metadata for a single signing certificate discovered on a hardware source (Windows certificate
* store or a PKCS#11 token). Returned to the desktop frontend so the user can pick which
* certificate to sign with. Never carries private key material - signing always happens on the
* token / OS.
*/
public record HardwareCertificateInfo(
String alias,
String source,
String subject,
String issuer,
String subjectCommonName,
String issuerCommonName,
String serialNumber,
String keyAlgorithm,
String notBefore,
String notAfter,
boolean expired,
boolean notYetValid) {}
@@ -0,0 +1,19 @@
package stirling.software.SPDF.model.api.security;
import java.util.List;
/**
* Describes what hardware-backed signing the local backend can offer. Only meaningful on the
* desktop bundle, where the backend runs as a local sidecar in the signed-in user's session and can
* reach the Windows certificate store / a plugged-in USB PKCS#11 token.
*/
public record HardwareSigningCapabilities(
boolean desktop,
String osName,
boolean windowsStoreSupported,
boolean pkcs11Supported,
List<Pkcs11LibraryInfo> detectedLibraries) {
/** A PKCS#11 driver library detected on disk (or supplied via configuration). */
public record Pkcs11LibraryInfo(String name, String path) {}
}
@@ -0,0 +1,7 @@
package stirling.software.SPDF.model.api.security;
/**
* Request body for enumerating the certificates on a PKCS#11 token. The PIN is required to log into
* the token; it is used only for the duration of the call and never stored.
*/
public record Pkcs11CertificatesRequest(String libraryPath, Integer slot, String pin) {}
@@ -14,8 +14,10 @@ import stirling.software.common.model.api.PDFFile;
public class SignPDFWithCertRequest extends PDFFile {
@Schema(
description = "The type of the digital certificate",
allowableValues = {"PEM", "PKCS12", "PFX", "JKS", "SERVER"},
description =
"The type of the digital certificate. WINDOWS_STORE and PKCS11 are"
+ " hardware-backed and only available in the desktop app.",
allowableValues = {"PEM", "PKCS12", "PFX", "JKS", "SERVER", "WINDOWS_STORE", "PKCS11"},
requiredMode = Schema.RequiredMode.REQUIRED)
private String certType;
@@ -39,9 +41,31 @@ public class SignPDFWithCertRequest extends PDFFile {
@Schema(description = "The JKS keystore file (Java Key Store)")
private MultipartFile jksFile;
@Schema(description = "The password for the keystore or the private key", format = "password")
@Schema(
description =
"The password for the keystore / private key, or the token PIN for PKCS11",
format = "password")
private String password;
@Schema(
description =
"The alias of the certificate to sign with. Required for WINDOWS_STORE and"
+ " recommended for PKCS11 tokens holding multiple certificates.")
private String alias;
@Schema(
description =
"Absolute path to the PKCS#11 driver library (required for PKCS11 type). Must"
+ " be an allowed driver - a detected one or configured via"
+ " STIRLING_PKCS11_LIBRARIES.")
private String pkcs11LibraryPath;
@Schema(
description =
"Optional PKCS#11 slot index. When omitted the first slot with a token is"
+ " used.")
private Integer pkcs11Slot;
@Schema(
description = "Whether to visually show the signature in the PDF file",
defaultValue = "false",
@@ -18,6 +18,11 @@ public class SignatureValidationResult {
// Time validation
private boolean notExpired;
// Whether the document's signatures cover all of its bytes. False when content was appended
// outside every signature's ByteRange (i.e. added after signing), which the signature can't
// attest to even though the signed bytes themselves remain cryptographically intact.
private boolean coversEntireDocument = true;
// Revocation validation
private boolean revocationChecked; // true if PKIX revocation was enabled
private String revocationStatus; // "not-checked" | "good" | "revoked" | "soft-fail" | "unknown"
@@ -115,7 +115,8 @@ public class CertificateValidationService {
log.info("Enabled AIA certificate fetching and revocation checking");
}
// Trust only what we explicitly opt into:
// Trust only what we explicitly opt into. Desktop follows the same flags as the server -
// our own signing cert is trusted via serverAsAnchor, not by force-loading every system CA.
if (validation.getTrust().isServerAsAnchor()) loadServerCertAsAnchor();
if (validation.getTrust().isUseSystemTrust()) loadJavaSystemTrustStore();
if (validation.getTrust().isUseMozillaBundle()) loadBundledMozillaCACerts();
@@ -0,0 +1,483 @@
package stirling.software.SPDF.service;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyStore;
import java.security.Provider;
import java.security.Security;
import java.security.cert.Certificate;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import javax.security.auth.x500.X500Principal;
import org.bouncycastle.asn1.x500.RDN;
import org.bouncycastle.asn1.x500.X500Name;
import org.bouncycastle.asn1.x500.style.BCStyle;
import org.bouncycastle.asn1.x500.style.IETFUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.security.HardwareCertificateInfo;
import stirling.software.SPDF.model.api.security.HardwareSigningCapabilities;
import stirling.software.SPDF.model.api.security.HardwareSigningCapabilities.Pkcs11LibraryInfo;
import stirling.software.common.util.ExceptionUtils;
/**
* Bridges PDF signing to hardware-held keys: the Windows certificate store (via the JDK SunMSCAPI
* provider) and USB / smart-card PKCS#11 tokens (via SunPKCS11). The private key never leaves the
* token - the JCA routes the actual signing operation onto the hardware.
*
* <p>These code paths are gated to the desktop bundle. On a hosted server the backend cannot reach
* a remote user's USB token anyway, and loading an arbitrary PKCS#11 driver library is effectively
* native code execution, so PKCS#11 libraries are additionally restricted to an allowlist of
* detected / configured driver paths.
*/
@Service
@Slf4j
public class HardwareKeyStoreService {
public static final String SOURCE_WINDOWS_STORE = "WINDOWS_STORE";
public static final String SOURCE_PKCS11 = "PKCS11";
private static final String WINDOWS_KEYSTORE_TYPE = "Windows-MY";
private static final String MSCAPI_PROVIDER = "SunMSCAPI";
private static final String PKCS11_BASE_PROVIDER = "SunPKCS11";
/** Extra PKCS#11 driver libraries, absolute paths, comma/`File.pathSeparator` separated. */
private static final String PKCS11_LIBRARIES_ENV = "STIRLING_PKCS11_LIBRARIES";
/** Same as {@link #PKCS11_LIBRARIES_ENV} but as a JVM system property. */
private static final String PKCS11_LIBRARIES_PROP = "stirling.pkcs11.libraries";
private final String machineType;
public HardwareKeyStoreService(
@Autowired(required = false) @Qualifier("machineType") String machineType) {
this.machineType = machineType;
}
// ---------------------------------------------------------------------
// Gating
// ---------------------------------------------------------------------
/**
* True when running as the desktop bundle (local sidecar in the user's session). The Tauri
* bundle sets {@code STIRLING_PDF_TAURI_MODE=true} (with {@code BROWSER_OPEN=false}, so
* machineType is {@code Server-jar} there); the bare-jar desktop launcher instead yields a
* {@code Client-*} machineType. Accept either.
*/
public boolean isDesktop() {
if (Boolean.parseBoolean(System.getProperty("STIRLING_PDF_TAURI_MODE", "false"))) {
return true;
}
return machineType != null && machineType.startsWith("Client-");
}
public boolean isWindows() {
return System.getProperty("os.name", "").toLowerCase(Locale.ROOT).contains("win");
}
private boolean windowsStoreSupported() {
return isWindows() && Security.getProvider(MSCAPI_PROVIDER) != null;
}
private boolean pkcs11Supported() {
return Security.getProvider(PKCS11_BASE_PROVIDER) != null;
}
/** Reject anything that is not the desktop bundle reached over loopback. */
public void assertLocalDesktop(HttpServletRequest request) {
if (!isDesktop()) {
throw ExceptionUtils.createIllegalArgumentException(
"error.hardwareSigningDesktopOnly",
"Hardware-backed signing is only available in the Stirling PDF desktop app");
}
if (request != null && !isLocalRequest(request.getRemoteAddr())) {
throw ExceptionUtils.createIllegalArgumentException(
"error.hardwareSigningLocalOnly",
"Hardware-backed signing can only be used from this device");
}
}
/**
* True when the request originates from this machine. Loopback (incl. IPv4-mapped IPv6 like
* {@code ::ffff:127.0.0.1}) counts, as does any address bound to a local interface - so it
* works whether the desktop app reaches the sidecar over {@code localhost} or a LAN IP, while
* still rejecting other machines on the network.
*/
static boolean isLocalRequest(String remoteAddr) {
if (remoteAddr == null || remoteAddr.isBlank()) {
return false;
}
try {
InetAddress addr = InetAddress.getByName(remoteAddr);
if (addr.isLoopbackAddress() || addr.isAnyLocalAddress()) {
return true;
}
return NetworkInterface.networkInterfaces()
.anyMatch(nif -> nif.inetAddresses().anyMatch(local -> local.equals(addr)));
} catch (Exception e) {
return false;
}
}
// ---------------------------------------------------------------------
// Capabilities
// ---------------------------------------------------------------------
public HardwareSigningCapabilities capabilities() {
boolean desktop = isDesktop();
if (!desktop) {
return new HardwareSigningCapabilities(false, "", false, false, List.of());
}
return new HardwareSigningCapabilities(
true,
System.getProperty("os.name", ""),
windowsStoreSupported(),
pkcs11Supported(),
detectPkcs11Libraries());
}
/**
* Known driver install locations plus any paths configured via {@code
* STIRLING_PKCS11_LIBRARIES}.
*/
public List<Pkcs11LibraryInfo> detectPkcs11Libraries() {
Map<String, List<String>> candidates = new LinkedHashMap<>();
String os = System.getProperty("os.name", "").toLowerCase(Locale.ROOT);
if (os.contains("win")) {
candidates.put(
"OpenSC",
List.of(
"C:\\Program Files\\OpenSC Project\\OpenSC\\pkcs11\\opensc-pkcs11.dll"));
candidates.put(
"YubiKey (ykcs11)",
List.of("C:\\Program Files\\Yubico\\Yubico PIV Tool\\bin\\libykcs11.dll"));
candidates.put("SafeNet eToken", List.of("C:\\Windows\\System32\\eTPKCS11.dll"));
candidates.put(
"Thales/Gemalto IDPrime", List.of("C:\\Windows\\System32\\IDPrimePKCS11.dll"));
candidates.put(
"SoftHSM2",
List.of(
"C:\\Program Files\\SoftHSM2\\lib\\softhsm2-x64.dll",
"C:\\SoftHSM2\\lib\\softhsm2-x64.dll"));
} else if (os.contains("mac")) {
candidates.put(
"OpenSC",
List.of(
"/Library/OpenSC/lib/opensc-pkcs11.so",
"/usr/local/lib/opensc-pkcs11.so"));
candidates.put(
"YubiKey (ykcs11)",
List.of("/usr/local/lib/libykcs11.dylib", "/opt/homebrew/lib/libykcs11.dylib"));
candidates.put(
"SoftHSM2",
List.of(
"/usr/local/lib/softhsm/libsofthsm2.so",
"/opt/homebrew/lib/softhsm/libsofthsm2.so"));
} else {
candidates.put(
"OpenSC",
List.of(
"/usr/lib/x86_64-linux-gnu/opensc-pkcs11.so",
"/usr/lib/opensc-pkcs11.so",
"/usr/lib64/opensc-pkcs11.so"));
candidates.put(
"YubiKey (ykcs11)",
List.of(
"/usr/lib/x86_64-linux-gnu/libykcs11.so",
"/usr/local/lib/libykcs11.so"));
candidates.put(
"SoftHSM2",
List.of(
"/usr/lib/softhsm/libsofthsm2.so",
"/usr/lib64/softhsm/libsofthsm2.so",
"/usr/local/lib/softhsm/libsofthsm2.so"));
}
List<Pkcs11LibraryInfo> result = new ArrayList<>();
candidates.forEach(
(name, paths) ->
paths.stream()
.filter(p -> Files.exists(Path.of(p)))
.findFirst()
.ifPresent(p -> result.add(new Pkcs11LibraryInfo(name, p))));
for (String configured : configuredLibraries()) {
if (Files.exists(Path.of(configured))
&& result.stream().noneMatch(l -> sameFile(l.path(), configured))) {
result.add(new Pkcs11LibraryInfo(fileName(configured), configured));
}
}
return result;
}
private static List<String> configuredLibraries() {
String env = System.getenv(PKCS11_LIBRARIES_ENV);
String prop = System.getProperty(PKCS11_LIBRARIES_PROP);
StringBuilder combined = new StringBuilder();
if (env != null && !env.isBlank()) {
combined.append(env);
}
if (prop != null && !prop.isBlank()) {
if (combined.length() > 0) {
combined.append(java.io.File.pathSeparator);
}
combined.append(prop);
}
if (combined.length() == 0) {
return List.of();
}
return Arrays.stream(combined.toString().split("[,;" + java.io.File.pathSeparator + "]"))
.map(String::trim)
.filter(s -> !s.isEmpty())
.toList();
}
// ---------------------------------------------------------------------
// Windows certificate store
// ---------------------------------------------------------------------
public KeyStore loadWindowsKeyStore() throws Exception {
if (!windowsStoreSupported()) {
throw ExceptionUtils.createIllegalArgumentException(
"error.windowsStoreUnavailable",
"The Windows certificate store is not available on this platform");
}
KeyStore ks = KeyStore.getInstance(WINDOWS_KEYSTORE_TYPE, MSCAPI_PROVIDER);
ks.load(null, null);
return ks;
}
public Provider windowsProvider() {
return Security.getProvider(MSCAPI_PROVIDER);
}
public List<HardwareCertificateInfo> listWindowsCertificates() throws Exception {
return listSigningCertificates(loadWindowsKeyStore(), SOURCE_WINDOWS_STORE);
}
// ---------------------------------------------------------------------
// PKCS#11 tokens
// ---------------------------------------------------------------------
/**
* A configured, logged-in PKCS#11 keystore plus the provider that must service signing. Closing
* logs the session out so the PIN-authenticated session does not outlive the request. The
* provider stays cached (logout is C_Logout, not C_Finalize) so the next call reuses the same
* C_Initialize. Single-user desktop model - logout is best-effort.
*/
public record Pkcs11Session(KeyStore keyStore, Provider provider) implements AutoCloseable {
@Override
public void close() {
if (provider instanceof java.security.AuthProvider authProvider) {
try {
authProvider.logout();
} catch (Exception e) {
// Not logged in / already logged out - nothing to clear.
}
}
}
}
// One SunPKCS11 provider per driver+slot, reused across enumerate + sign. A PKCS#11 module
// typically allows C_Initialize only once per process, so configuring a fresh provider on every
// call races with the previous (not-yet-GC'd) one - the cause of "first sign fails, second
// works". Reusing the provider keeps a single C_Initialize alive for the session.
private final java.util.concurrent.ConcurrentHashMap<String, Provider> pkcs11Providers =
new java.util.concurrent.ConcurrentHashMap<>();
public Pkcs11Session openPkcs11(String libraryPath, Integer slot, char[] pin) throws Exception {
validateLibraryAllowed(libraryPath);
if (!pkcs11Supported()) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pkcs11Unavailable", "PKCS#11 support is not available in this runtime");
}
String cacheKey = libraryPath + "|" + slot;
Provider provider =
pkcs11Providers.computeIfAbsent(
cacheKey, k -> buildPkcs11Provider(libraryPath, slot));
try {
KeyStore ks = KeyStore.getInstance("PKCS11", provider);
ks.load(null, pin);
return new Pkcs11Session(ks, provider);
} catch (Exception e) {
// A wrong PIN must not be retried: a second C_Login would burn the token's retry
// counter twice per attempt and can lock the token. Only rebuild on provider/init
// failures (e.g. token removed/re-inserted leaving a stale provider).
if (isAuthFailure(e)) {
throw e;
}
pkcs11Providers.remove(cacheKey, provider);
Provider fresh =
pkcs11Providers.computeIfAbsent(
cacheKey, k -> buildPkcs11Provider(libraryPath, slot));
KeyStore ks = KeyStore.getInstance("PKCS11", fresh);
ks.load(null, pin);
return new Pkcs11Session(ks, fresh);
}
}
/** True when the failure is a bad/locked PIN rather than a provider/init/device problem. */
private static boolean isAuthFailure(Throwable t) {
while (t != null) {
if (t instanceof javax.security.auth.login.FailedLoginException) {
return true;
}
String msg = t.getMessage();
if (msg != null && msg.toUpperCase(Locale.ROOT).contains("CKR_PIN")) {
return true; // CKR_PIN_INCORRECT / CKR_PIN_LOCKED / CKR_PIN_INVALID / ...
}
t = t.getCause();
}
return false;
}
private Provider buildPkcs11Provider(String libraryPath, Integer slot) {
StringBuilder config = new StringBuilder();
config.append("--name=").append(providerName(libraryPath)).append('\n');
config.append("library=").append(libraryPath).append('\n');
if (slot != null) {
config.append("slot=").append(slot).append('\n');
}
try {
return Security.getProvider(PKCS11_BASE_PROVIDER).configure(config.toString());
} catch (Exception e) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pkcs11ConfigFailed",
"Failed to initialise the PKCS#11 driver: {0}",
e.getMessage());
}
}
public List<HardwareCertificateInfo> listPkcs11Certificates(
String libraryPath, Integer slot, char[] pin) throws Exception {
try (Pkcs11Session session = openPkcs11(libraryPath, slot, pin)) {
return listSigningCertificates(session.keyStore(), SOURCE_PKCS11);
}
}
/**
* Reject driver paths that are not detected on disk / configured - blocks arbitrary DLL loads.
*/
public void validateLibraryAllowed(String libraryPath) {
if (libraryPath == null || libraryPath.isBlank()) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pkcs11LibraryRequired", "A PKCS#11 driver library path is required");
}
Set<String> allowed =
detectPkcs11Libraries().stream()
.map(Pkcs11LibraryInfo::path)
.collect(Collectors.toSet());
boolean ok = allowed.stream().anyMatch(p -> sameFile(p, libraryPath));
if (!ok) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pkcs11LibraryNotAllowed",
"PKCS#11 driver is not in the allowed list. Add it via the"
+ " STIRLING_PKCS11_LIBRARIES setting: {0}",
libraryPath);
}
}
// ---------------------------------------------------------------------
// Shared helpers
// ---------------------------------------------------------------------
private List<HardwareCertificateInfo> listSigningCertificates(KeyStore ks, String source)
throws Exception {
List<HardwareCertificateInfo> certs = new ArrayList<>();
Enumeration<String> aliases = ks.aliases();
while (aliases.hasMoreElements()) {
String alias = aliases.nextElement();
if (!ks.isKeyEntry(alias)) {
continue; // only entries we can sign with
}
Certificate cert = ks.getCertificate(alias);
if (cert instanceof X509Certificate x509) {
certs.add(toInfo(alias, x509, source));
}
}
return certs;
}
private static HardwareCertificateInfo toInfo(
String alias, X509Certificate cert, String source) {
java.util.Date now = new java.util.Date();
return new HardwareCertificateInfo(
alias,
source,
cert.getSubjectX500Principal().getName(),
cert.getIssuerX500Principal().getName(),
commonName(cert.getSubjectX500Principal()),
commonName(cert.getIssuerX500Principal()),
cert.getSerialNumber().toString(16),
cert.getPublicKey().getAlgorithm(),
cert.getNotBefore().toInstant().toString(),
cert.getNotAfter().toInstant().toString(),
now.after(cert.getNotAfter()),
now.before(cert.getNotBefore()));
}
private static String commonName(X500Principal principal) {
try {
X500Name x500Name = new X500Name(principal.getName());
RDN[] rdns = x500Name.getRDNs(BCStyle.CN);
if (rdns.length > 0) {
return IETFUtils.valueToString(rdns[0].getFirst().getValue());
}
} catch (Exception e) {
log.debug("Could not parse common name from {}", principal.getName());
}
return principal.getName();
}
private static String providerName(String libraryPath) {
String base = fileName(libraryPath).replaceAll("[^a-zA-Z0-9]", "");
if (base.isEmpty()) {
base = "token";
}
return "StirlingHW" + base;
}
private static String fileName(String path) {
try {
return Path.of(path).getFileName().toString();
} catch (Exception e) {
return path;
}
}
private static boolean sameFile(String a, String b) {
if (a == null || b == null) {
return false;
}
try {
Path pa = Path.of(a);
Path pb = Path.of(b);
if (Files.exists(pa) && Files.exists(pb)) {
return Files.isSameFile(pa, pb);
}
return pa.toAbsolutePath().normalize().equals(pb.toAbsolutePath().normalize());
} catch (Exception e) {
return a.equalsIgnoreCase(b);
}
}
}
@@ -179,6 +179,13 @@
"moduleLicense": "Apache-2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "com.google.code.gson:gson",
"moduleUrl": "https://github.com/google/gson",
"moduleVersion": "2.14.0",
"moduleLicense": "Apache-2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "com.google.errorprone:error_prone_annotations",
"moduleUrl": "https://errorprone.info/error_prone_annotations",
@@ -186,6 +193,13 @@
"moduleLicense": "Apache 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "com.google.errorprone:error_prone_annotations",
"moduleUrl": "https://errorprone.info/error_prone_annotations",
"moduleVersion": "2.48.0",
"moduleLicense": "Apache 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "com.google.guava:failureaccess",
"moduleUrl": "https://github.com/google/guava/",
@@ -629,13 +643,6 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "commons-io:commons-io",
"moduleUrl": "https://commons.apache.org/proper/commons-io/",
"moduleVersion": "2.21.0",
"moduleLicense": "Apache-2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "commons-io:commons-io",
"moduleUrl": "https://commons.apache.org/proper/commons-io/",
@@ -9,6 +9,7 @@ import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import org.apache.pdfbox.Loader;
@@ -302,6 +303,11 @@ class RearrangePagesPDFControllerTest {
assertNotNull(response);
// 2 pages * 3 duplicates = 6 final pages
assertEquals(6, realDoc.getNumberOfPages());
// Each duplicate must be a distinct page node in the saved output; a shared
// node under multiple /Kids is an invalid tree readers reject as cyclic.
List<Object> savedPages = reloadAndSnapshot(response);
assertEquals(6, savedPages.size());
assertEquals(6, new HashSet<>(savedPages).size());
}
}
@@ -323,4 +329,29 @@ class RearrangePagesPDFControllerTest {
assertEquals(4, realDoc.getNumberOfPages());
}
}
@Test
void testRearrangePages_SideStitchBooklet_RepeatedPaddingPagesAreDistinctNodes()
throws IOException {
MockMultipartFile file = createMockPdf();
RearrangePagesRequest request = new RearrangePagesRequest();
request.setFileInput(file);
request.setPageNumbers("");
request.setCustomMode("SIDE_STITCH_BOOKLET_SORT");
// 6 pages is not a multiple of 4, so booklet padding repeats the last page index
// several times; each repeat must be a distinct page node, not one shared node.
try (PDDocument realDoc = buildRealPdf(6)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertEquals(8, realDoc.getNumberOfPages());
List<Object> savedPages = reloadAndSnapshot(response);
assertEquals(8, savedPages.size());
assertEquals(8, new HashSet<>(savedPages).size());
}
}
}
@@ -30,7 +30,10 @@ import org.springframework.http.ResponseEntity;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.SPDF.model.api.security.SignPDFWithCertRequest;
import stirling.software.SPDF.service.HardwareKeyStoreService;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
@@ -51,6 +54,8 @@ class CertSignControllerTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
@Mock private TempFileManager tempFileManager;
@Mock private HardwareKeyStoreService hardwareKeyStoreService;
@Mock private HttpServletRequest httpRequest;
@InjectMocks private CertSignController certSignController;
@@ -169,7 +174,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -195,7 +201,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -221,7 +228,7 @@ class CertSignControllerTest {
IllegalArgumentException exception =
assertThrows(
IllegalArgumentException.class,
() -> certSignController.signPDFWithCert(request));
() -> certSignController.signPDFWithCert(request, httpRequest));
assertTrue(exception.getMessage().contains("PKCS12 keystore"));
}
@@ -247,7 +254,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -278,7 +286,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -309,7 +318,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -340,7 +350,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -371,7 +382,8 @@ class CertSignControllerTest {
request.setPageNumber(1);
request.setShowLogo(false);
ResponseEntity<Resource> response = certSignController.signPDFWithCert(request);
ResponseEntity<Resource> response =
certSignController.signPDFWithCert(request, httpRequest);
assertNotNull(response.getBody());
assertTrue(drainBody(response).length > 0);
@@ -0,0 +1,146 @@
package stirling.software.SPDF.service;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.SPDF.model.api.security.HardwareSigningCapabilities;
/** Unit tests for the gating / allowlist logic that protects the hardware signing paths. */
class HardwareKeyStoreServiceTest {
private static final String PKCS11_PROP = "stirling.pkcs11.libraries";
private HardwareKeyStoreService service(String machineType) {
return new HardwareKeyStoreService(machineType);
}
@Test
void isDesktop_trueOnlyForClientMachineTypes() {
assertTrue(service("Client-windows").isDesktop());
assertTrue(service("Client-mac").isDesktop());
assertTrue(service("Client-unix").isDesktop());
assertFalse(service("Server-jar").isDesktop());
assertFalse(service("Docker").isDesktop());
assertFalse(service(null).isDesktop());
}
@Test
void isDesktop_trueInTauriModeEvenWithoutClientMachineType() {
// The Tauri bundle sets STIRLING_PDF_TAURI_MODE=true while machineType stays Server-jar.
String previous = System.getProperty("STIRLING_PDF_TAURI_MODE");
try {
System.setProperty("STIRLING_PDF_TAURI_MODE", "true");
assertTrue(service("Server-jar").isDesktop());
assertTrue(service(null).isDesktop());
} finally {
if (previous == null) {
System.clearProperty("STIRLING_PDF_TAURI_MODE");
} else {
System.setProperty("STIRLING_PDF_TAURI_MODE", previous);
}
}
}
@Test
void capabilities_notDesktop_reportsUnavailable() {
HardwareSigningCapabilities caps = service("Server-jar").capabilities();
assertFalse(caps.desktop());
assertFalse(caps.windowsStoreSupported());
assertFalse(caps.pkcs11Supported());
assertTrue(caps.detectedLibraries().isEmpty());
}
@Test
void capabilities_desktop_reportsOsName() {
HardwareSigningCapabilities caps = service("Client-windows").capabilities();
assertTrue(caps.desktop());
assertFalse(caps.osName().isBlank());
}
@Test
void assertLocalDesktop_rejectsNonDesktop() {
HttpServletRequest request = mock(HttpServletRequest.class);
when(request.getRemoteAddr()).thenReturn("127.0.0.1");
assertThrows(
IllegalArgumentException.class,
() -> service("Server-jar").assertLocalDesktop(request));
}
@Test
void assertLocalDesktop_rejectsRemoteCallerEvenOnDesktop() {
HttpServletRequest request = mock(HttpServletRequest.class);
// 203.0.113.0/24 is TEST-NET-3 (RFC 5737) - never a real local interface address.
when(request.getRemoteAddr()).thenReturn("203.0.113.5");
assertThrows(
IllegalArgumentException.class,
() -> service("Client-windows").assertLocalDesktop(request));
}
@Test
void assertLocalDesktop_allowsLoopbackOnDesktop() {
HttpServletRequest request = mock(HttpServletRequest.class);
when(request.getRemoteAddr()).thenReturn("127.0.0.1");
assertDoesNotThrow(() -> service("Client-windows").assertLocalDesktop(request));
// No servlet context (e.g. internal call) is also allowed.
assertDoesNotThrow(() -> service("Client-windows").assertLocalDesktop(null));
}
@Test
void isLocalRequest_acceptsLoopbackForms_rejectsRemote() {
assertTrue(HardwareKeyStoreService.isLocalRequest("127.0.0.1"));
assertTrue(HardwareKeyStoreService.isLocalRequest("::1"));
assertTrue(HardwareKeyStoreService.isLocalRequest("0:0:0:0:0:0:0:1"));
// IPv4-mapped IPv6 loopback - what Tomcat reports for the desktop webview.
assertTrue(HardwareKeyStoreService.isLocalRequest("::ffff:127.0.0.1"));
assertFalse(HardwareKeyStoreService.isLocalRequest("203.0.113.5"));
assertFalse(HardwareKeyStoreService.isLocalRequest(null));
}
@Test
void validateLibraryAllowed_blankPath_throws() {
assertThrows(
IllegalArgumentException.class,
() -> service("Client-windows").validateLibraryAllowed(" "));
}
@Test
void validateLibraryAllowed_unknownPath_throws() {
assertThrows(
IllegalArgumentException.class,
() ->
service("Client-windows")
.validateLibraryAllowed("/definitely/not/a/real/driver.so"));
}
@Test
void validateLibraryAllowed_configuredPath_isAllowed(@TempDir Path tempDir) throws Exception {
Path fakeDriver = Files.createFile(tempDir.resolve("fake-pkcs11.so"));
String previous = System.getProperty(PKCS11_PROP);
try {
System.setProperty(PKCS11_PROP, fakeDriver.toString());
HardwareKeyStoreService service = service("Client-windows");
assertDoesNotThrow(() -> service.validateLibraryAllowed(fakeDriver.toString()));
assertTrue(
service.detectPkcs11Libraries().stream()
.anyMatch(l -> l.path().equals(fakeDriver.toString())));
} finally {
if (previous == null) {
System.clearProperty(PKCS11_PROP);
} else {
System.setProperty(PKCS11_PROP, previous);
}
}
}
}
+4
View File
@@ -80,6 +80,10 @@ dependencies {
implementation "software.amazon.awssdk:s3:${awsSdkVersion}"
implementation "software.amazon.awssdk:url-connection-client:${awsSdkVersion}"
// @DataJpaTest slice (Boot 4 ships test slices as separate starters, like webmvc-test at the
// root) so policy.source repositories can be exercised against embedded H2.
testImplementation 'org.springframework.boot:spring-boot-starter-data-jpa-test'
// Testcontainers: real MinIO/LocalStack (S3) and Valkey for integration tests in CI without
// manually-started instances. Tests skip cleanly when Docker is unavailable.
testImplementation "org.testcontainers:testcontainers:${testcontainersMinioVersion}"
@@ -0,0 +1,20 @@
package stirling.software.proprietary.access.config;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.proprietary.access.service.DefaultTeamLeadLookup;
import stirling.software.proprietary.access.service.TeamLeadLookup;
/** Access-layer bean wiring. */
@Configuration
public class AccessConfig {
/** No-op {@link TeamLeadLookup} unless another bean is defined. */
@Bean
@ConditionalOnMissingBean(TeamLeadLookup.class)
TeamLeadLookup defaultTeamLeadLookup() {
return new DefaultTeamLeadLookup();
}
}
@@ -0,0 +1,101 @@
package stirling.software.proprietary.access.controller;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.User;
/** Admin endpoints to grant/revoke access to gated resources (the portal, integration configs). */
@RestController
@RequestMapping("/api/v1/admin/access")
@RequiredArgsConstructor
@PreAuthorize("hasRole('ADMIN')")
@Tag(name = "Access Control", description = "Manage resource access grants (portal, integrations)")
public class ResourceGrantController {
private final ResourceAccessService accessService;
@GetMapping("/grants")
public ResponseEntity<?> list(
@RequestParam ResourceType resourceType,
@RequestParam(required = false, defaultValue = "") String resourceId) {
List<ResourceGrant> grants = accessService.listGrants(resourceType, resourceId);
return ResponseEntity.ok(grants.stream().map(this::toDto).toList());
}
@PostMapping("/grants")
public ResponseEntity<?> create(
@RequestBody GrantRequest request, @AuthenticationPrincipal User admin) {
if (request.resourceType() == null
|| request.principalType() == null
|| request.principalId() == null) {
return ResponseEntity.badRequest()
.body(
Map.of(
"error",
"resourceType, principalType and principalId are required"));
}
AccessPermission permission =
request.permission() == null ? AccessPermission.USE : request.permission();
// PORTAL is a singleton resource; its grants always target the whole type.
String resourceId =
request.resourceType() == ResourceType.PORTAL ? "" : request.resourceId();
ResourceGrant grant =
accessService.grant(
request.resourceType(),
resourceId,
request.principalType(),
request.principalId(),
permission,
admin);
return ResponseEntity.ok(toDto(grant));
}
@DeleteMapping("/grants/{id}")
public ResponseEntity<?> delete(@PathVariable Long id) {
accessService.revoke(id);
return ResponseEntity.ok(Map.of("message", "Grant revoked"));
}
private Map<String, Object> toDto(ResourceGrant g) {
Map<String, Object> m = new HashMap<>();
m.put("id", g.getId());
m.put("resourceType", g.getResourceType());
m.put("resourceId", g.getResourceId());
m.put("principalType", g.getPrincipalType());
m.put("principalId", g.getPrincipalId());
m.put("permission", g.getPermission());
m.put("createdAt", g.getCreatedAt());
return m;
}
/** Request body for creating a grant. */
public record GrantRequest(
ResourceType resourceType,
String resourceId,
PrincipalType principalType,
Long principalId,
AccessPermission permission) {}
}
@@ -0,0 +1,7 @@
package stirling.software.proprietary.access.model;
/** Permission level a grant confers. MANAGE implies USE. */
public enum AccessPermission {
USE,
MANAGE
}
@@ -0,0 +1,14 @@
package stirling.software.proprietary.access.model;
/**
* Fallback policy applied when no explicit {@link ResourceGrant} matches. Admins (org owners)
* always pass regardless of this policy.
*/
public enum DefaultAccessPolicy {
// Every authenticated user in the deployment (org) may use the resource.
ORG_ALL,
// Only org admins and team leaders. This is the default for the portal.
ADMINS_AND_TEAM_LEADS,
// Nobody but the owner, admins, and explicit grantees.
EXPLICIT_ONLY
}
@@ -0,0 +1,57 @@
package stirling.software.proprietary.access.model;
import jakarta.persistence.Column;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.FetchType;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.MappedSuperclass;
import lombok.Getter;
import lombok.Setter;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
/** Base for a resource owned by a user, a team, or the server, with grant-based access. */
@MappedSuperclass
@Getter
@Setter
public abstract class OwnedResource {
@Enumerated(EnumType.STRING)
@Column(name = "scope", nullable = false, length = 32)
private OwnerScope scope;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "owner_user_id")
private User ownerUser;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "owner_team_id")
private Team ownerTeam;
@Column(name = "enabled", nullable = false)
private boolean enabled = true;
// Server resource that users cannot override with their own of the same kind.
@Column(name = "locked", nullable = false)
private boolean locked = false;
// Who, besides owner/admin/grantees, may use this resource.
@Enumerated(EnumType.STRING)
@Column(name = "default_access", nullable = false, length = 32)
private DefaultAccessPolicy defaultAccess = DefaultAccessPolicy.EXPLICIT_ONLY;
/** Subclass primary key. */
public abstract Long getId();
public Long getOwnerUserId() {
return ownerUser != null ? ownerUser.getId() : null;
}
public Long getOwnerTeamId() {
return ownerTeam != null ? ownerTeam.getId() : null;
}
}
@@ -0,0 +1,8 @@
package stirling.software.proprietary.access.model;
/** Ownership scope of an {@link OwnedResource}: a single user, a team, or the whole server. */
public enum OwnerScope {
USER,
TEAM,
SERVER
}
@@ -0,0 +1,7 @@
package stirling.software.proprietary.access.model;
/** Who a {@link ResourceGrant} is granted to. Org-wide access is expressed via default policy. */
public enum PrincipalType {
USER,
TEAM
}
@@ -0,0 +1,86 @@
package stirling.software.proprietary.access.model;
import java.io.Serializable;
import java.time.LocalDateTime;
import org.hibernate.annotations.CreationTimestamp;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Index;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.persistence.UniqueConstraint;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.security.model.User;
/** Grants a user or team access to a resource. Owner and admin access are implicit. */
@Entity
@Table(
name = "resource_grants",
uniqueConstraints =
@UniqueConstraint(
name = "uk_resource_grant",
columnNames = {
"resource_type",
"resource_id",
"principal_type",
"principal_id",
"permission"
}),
indexes = {
@Index(name = "idx_resource_grants_lookup", columnList = "resource_type,resource_id"),
@Index(
name = "idx_resource_grants_principal",
columnList = "principal_type,principal_id")
})
@NoArgsConstructor
@Getter
@Setter
public class ResourceGrant implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "resource_grant_id")
private Long id;
@Enumerated(EnumType.STRING)
@Column(name = "resource_type", nullable = false, length = 64)
private ResourceType resourceType;
// Empty string (never null) for a whole-type grant such as the portal.
@Column(name = "resource_id", nullable = false, length = 255)
private String resourceId = "";
@Enumerated(EnumType.STRING)
@Column(name = "principal_type", nullable = false, length = 32)
private PrincipalType principalType;
@Column(name = "principal_id", nullable = false)
private Long principalId;
@Enumerated(EnumType.STRING)
@Column(name = "permission", nullable = false, length = 32)
private AccessPermission permission;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "granted_by_user_id")
private User grantedBy;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
}
@@ -0,0 +1,9 @@
package stirling.software.proprietary.access.model;
/** Types of resources whose access can be gated by {@link ResourceGrant}. */
public enum ResourceType {
// The admin portal / processor (frontend/portal). Singleton resource (empty resourceId).
PORTAL,
// A stored S3/MCP/API integration configuration.
INTEGRATION_CONFIG
}
@@ -0,0 +1,28 @@
package stirling.software.proprietary.access.repository;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
@Repository
public interface ResourceGrantRepository extends JpaRepository<ResourceGrant, Long> {
List<ResourceGrant> findByResourceTypeAndResourceId(
ResourceType resourceType, String resourceId);
List<ResourceGrant> findByResourceTypeAndPrincipalTypeAndPrincipalId(
ResourceType resourceType, PrincipalType principalType, Long principalId);
void deleteByResourceTypeAndResourceId(ResourceType resourceType, String resourceId);
boolean existsByResourceTypeAndResourceIdAndPrincipalTypeAndPrincipalId(
ResourceType resourceType,
String resourceId,
PrincipalType principalType,
Long principalId);
}
@@ -0,0 +1,44 @@
package stirling.software.proprietary.access.security;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
/** {@code @PreAuthorize} bean for portal-access checks. Active in self-hosted and saas. */
@Component("resourceAccess")
@RequiredArgsConstructor
public class ResourceAccessSecurity {
private final ResourceAccessService accessService;
private final UserService userService;
public boolean canUsePortal() {
User user = currentUser();
return user != null && accessService.canAccessPortal(user);
}
private User currentUser() {
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
if (auth == null || !auth.isAuthenticated()) {
return null;
}
Object principal = auth.getPrincipal();
if (principal instanceof User user) {
return user;
}
if (principal instanceof UserDetails userDetails) {
return userService.findByUsername(userDetails.getUsername()).orElse(null);
}
if (principal instanceof String username && !"anonymousUser".equals(username)) {
return userService.findByUsername(username).orElse(null);
}
return null;
}
}
@@ -0,0 +1,17 @@
package stirling.software.proprietary.access.service;
import stirling.software.proprietary.security.model.User;
/** No-op {@link TeamLeadLookup}: always false. */
public class DefaultTeamLeadLookup implements TeamLeadLookup {
@Override
public boolean isAnyTeamLeader(User user) {
return false;
}
@Override
public boolean isLeaderOfTeam(User user, Long teamId) {
return false;
}
}
@@ -0,0 +1,116 @@
package stirling.software.proprietary.access.service;
import java.util.Set;
import java.util.function.BooleanSupplier;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.server.ResponseStatusException;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.TeamRepository;
/** Ownership and access checks for {@link OwnedResource}, backed by the resource-grant ACL. */
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class OwnershipService {
private final ResourceAccessService accessService;
private final TeamLeadLookup teamLeadLookup;
private final TeamRepository teamRepository;
/** Whether the user may use the resource. */
public boolean canUse(ResourceType type, OwnedResource resource, User user) {
if (!resource.isEnabled()) {
return isAdmin(user) || isOwner(resource, user);
}
return accessService.canUseResource(
type,
String.valueOf(resource.getId()),
resource.getOwnerUserId(),
resource.getDefaultAccess(),
user);
}
/** Whether the user may manage the resource. */
public boolean canManage(ResourceType type, OwnedResource resource, User user) {
return accessService.canManageResource(
type, String.valueOf(resource.getId()), resource.getOwnerUserId(), user);
}
/**
* Authorizes the scope and assigns ownership; {@code lockedOverrideBlocks} guards USER scope.
*/
public void assignOwnership(
OwnedResource resource,
OwnerScope scope,
Long teamId,
User user,
BooleanSupplier lockedOverrideBlocks) {
resource.setScope(scope);
switch (scope) {
case USER -> {
if (lockedOverrideBlocks.getAsBoolean() && !isAdmin(user)) {
throw forbidden(
"This is locked to the server configuration by an administrator");
}
resource.setOwnerUser(user);
}
case SERVER -> {
if (!isAdmin(user)) {
throw forbidden("Only administrators can create server-owned resources");
}
}
case TEAM -> {
if (teamId == null) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "ownerTeamId is required");
}
Team team =
teamRepository
.findById(teamId)
.orElseThrow(() -> notFound("Team not found"));
if (!isAdmin(user) && !teamLeadLookup.isLeaderOfTeam(user, team.getId())) {
throw forbidden("Only admins or team leaders can create team-owned resources");
}
resource.setOwnerTeam(team);
}
}
}
/** Resource ids of the given type the user or their team holds a grant on. */
public Set<String> grantedResourceIds(ResourceType type, User user) {
return accessService.grantedResourceIds(type, user);
}
public boolean isAdmin(User user) {
return user.getAuthorities().stream()
.anyMatch(a -> Role.ADMIN.getRoleId().equals(a.getAuthority()));
}
public boolean isOwner(OwnedResource resource, User user) {
if (resource.getOwnerUserId() != null && resource.getOwnerUserId().equals(user.getId())) {
return true;
}
// Team-owned: the lead of the owning team owns it.
return resource.getOwnerTeamId() != null
&& teamLeadLookup.isLeaderOfTeam(user, resource.getOwnerTeamId());
}
private ResponseStatusException forbidden(String message) {
return new ResponseStatusException(HttpStatus.FORBIDDEN, message);
}
private ResponseStatusException notFound(String message) {
return new ResponseStatusException(HttpStatus.NOT_FOUND, message);
}
}
@@ -0,0 +1,188 @@
package stirling.software.proprietary.access.service;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.repository.ResourceGrantRepository;
import stirling.software.proprietary.security.model.User;
/** Resolves access to gated resources: owner, then admin, then grant, then default policy. */
@Service
@RequiredArgsConstructor
@Slf4j
@Transactional(readOnly = true)
public class ResourceAccessService {
private final ResourceGrantRepository grantRepository;
private final TeamLeadLookup teamLeadLookup;
@Value("${security.portal.defaultAccess:ADMINS_AND_TEAM_LEADS}")
private DefaultAccessPolicy portalDefaultPolicy;
// ---- public checks ----
/** Whether the user may use the portal / processor. */
public boolean canAccessPortal(User user) {
return canUseResource(ResourceType.PORTAL, "", null, portalDefaultPolicy, user);
}
/** Whether the user may use a resource, falling back to its default policy. */
public boolean canUseResource(
ResourceType type,
String resourceId,
Long ownerUserId,
DefaultAccessPolicy defaultPolicy,
User user) {
if (user == null) {
return false;
}
if (isOwner(ownerUserId, user) || isAdmin(user)) {
return true;
}
if (hasGrant(type, normalize(resourceId), user, AccessPermission.USE)) {
return true;
}
return matchesDefault(defaultPolicy, user);
}
/** Whether the user may manage (edit/delete/share) a resource. No default-policy fallback. */
public boolean canManageResource(
ResourceType type, String resourceId, Long ownerUserId, User user) {
if (user == null) {
return false;
}
if (isOwner(ownerUserId, user) || isAdmin(user)) {
return true;
}
return hasGrant(type, normalize(resourceId), user, AccessPermission.MANAGE);
}
// ---- grant management ----
@Transactional
public ResourceGrant grant(
ResourceType type,
String resourceId,
PrincipalType principalType,
Long principalId,
AccessPermission permission,
User grantedBy) {
String rid = normalize(resourceId);
ResourceGrant grant =
grantRepository.findByResourceTypeAndResourceId(type, rid).stream()
.filter(
g ->
g.getPrincipalType() == principalType
&& g.getPrincipalId().equals(principalId))
.findFirst()
.orElseGet(ResourceGrant::new);
grant.setResourceType(type);
grant.setResourceId(rid);
grant.setPrincipalType(principalType);
grant.setPrincipalId(principalId);
grant.setPermission(permission);
if (grantedBy != null) {
grant.setGrantedBy(grantedBy);
}
return grantRepository.save(grant);
}
@Transactional
public void revoke(Long grantId) {
grantRepository.deleteById(grantId);
}
public List<ResourceGrant> listGrants(ResourceType type, String resourceId) {
return grantRepository.findByResourceTypeAndResourceId(type, normalize(resourceId));
}
/** Resource ids of the given type that this user (or their team) holds any grant on. */
public Set<String> grantedResourceIds(ResourceType type, User user) {
if (user == null) {
return Set.of();
}
Set<String> ids = new HashSet<>();
for (ResourceGrant g :
grantRepository.findByResourceTypeAndPrincipalTypeAndPrincipalId(
type, PrincipalType.USER, user.getId())) {
ids.add(g.getResourceId());
}
if (user.getTeam() != null) {
for (ResourceGrant g :
grantRepository.findByResourceTypeAndPrincipalTypeAndPrincipalId(
type, PrincipalType.TEAM, user.getTeam().getId())) {
ids.add(g.getResourceId());
}
}
return ids;
}
// ---- internals ----
private boolean hasGrant(
ResourceType type, String resourceId, User user, AccessPermission required) {
Long teamId = user.getTeam() != null ? user.getTeam().getId() : null;
for (ResourceGrant g : grantRepository.findByResourceTypeAndResourceId(type, resourceId)) {
if (!permissionSatisfies(g.getPermission(), required)) {
continue;
}
if (g.getPrincipalType() == PrincipalType.USER
&& g.getPrincipalId().equals(user.getId())) {
return true;
}
if (g.getPrincipalType() == PrincipalType.TEAM
&& teamId != null
&& g.getPrincipalId().equals(teamId)) {
return true;
}
}
return false;
}
// MANAGE implies USE.
private boolean permissionSatisfies(AccessPermission held, AccessPermission required) {
if (required == AccessPermission.USE) {
return held == AccessPermission.USE || held == AccessPermission.MANAGE;
}
return held == AccessPermission.MANAGE;
}
private boolean matchesDefault(DefaultAccessPolicy policy, User user) {
if (policy == null) {
return false;
}
return switch (policy) {
case ORG_ALL -> true;
// Admins already pass above; only team leads here.
case ADMINS_AND_TEAM_LEADS -> teamLeadLookup.isAnyTeamLeader(user);
case EXPLICIT_ONLY -> false;
};
}
private boolean isOwner(Long ownerUserId, User user) {
return ownerUserId != null && ownerUserId.equals(user.getId());
}
private boolean isAdmin(User user) {
return user.getAuthorities().stream()
.anyMatch(a -> Role.ADMIN.getRoleId().equals(a.getAuthority()));
}
private String normalize(String resourceId) {
return resourceId == null ? "" : resourceId;
}
}
@@ -0,0 +1,151 @@
package stirling.software.proprietary.access.service;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import org.springframework.stereotype.Component;
/** Masks, merges and sanitizes secret values in a config map, recursing into nested maps/lists. */
@Component
public class SecretMasker {
public static final String MASK = "********";
// Cap recursion so a pathologically nested payload cannot overflow the stack.
private static final int MAX_DEPTH = 32;
private static final Set<String> SENSITIVE_HINTS =
Set.of(
"secret",
"password",
"token",
"apikey",
"accesskey",
"credential",
"privatekey");
/** Replace sensitive values with the mask (recursively) for safe display. */
public Map<String, Object> mask(Map<String, Object> config) {
return mask(config, 0);
}
/** Drop sensitive blank/masked values from an incoming create payload. */
public Map<String, Object> sanitize(Map<String, Object> config) {
return sanitize(config, 0);
}
/**
* Merge an update over the stored map, keeping stored secrets where the incoming is redacted.
*/
public Map<String, Object> merge(Map<String, Object> stored, Map<String, Object> incoming) {
return merge(stored, incoming, 0);
}
private Map<String, Object> mask(Map<String, Object> config, int depth) {
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : config.entrySet()) {
out.put(e.getKey(), maskValue(e.getKey(), e.getValue(), depth));
}
return out;
}
private Map<String, Object> sanitize(Map<String, Object> config, int depth) {
if (config == null) {
return new LinkedHashMap<>();
}
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : config.entrySet()) {
if (isSensitive(e.getKey()) && isRedacted(e.getValue(), depth)) {
continue;
}
out.put(
e.getKey(),
e.getValue() instanceof Map<?, ?> m && depth < MAX_DEPTH
? sanitize(castMap(m), depth + 1)
: e.getValue());
}
return out;
}
private Map<String, Object> merge(
Map<String, Object> stored, Map<String, Object> incoming, int depth) {
Map<String, Object> out = new LinkedHashMap<>(stored);
for (Map.Entry<String, Object> e : incoming.entrySet()) {
String key = e.getKey();
Object value = e.getValue();
if (isSensitive(key)) {
if (!isRedacted(value, depth)) {
out.put(key, value); // a real new secret replaces the stored one
}
continue; // redacted (blank / mask) -> keep stored
}
if (depth < MAX_DEPTH
&& out.get(key) instanceof Map<?, ?> s
&& value instanceof Map<?, ?> i) {
out.put(key, merge(castMap(s), castMap(i), depth + 1));
} else {
out.put(key, value);
}
}
return out;
}
// A sensitive key masks its whole value; recurse into non-sensitive containers.
private Object maskValue(String key, Object value, int depth) {
if (isSensitive(key)) {
if (value == null || (value instanceof String s && s.isBlank())) {
return value;
}
return MASK;
}
if (depth >= MAX_DEPTH) {
// Too deep to descend; mask containers rather than risk leaking an unmasked secret.
return value instanceof Map<?, ?> || value instanceof List<?> ? MASK : value;
}
if (value instanceof Map<?, ?> m) {
return mask(castMap(m), depth + 1);
}
if (value instanceof List<?> list) {
List<Object> out = new ArrayList<>();
for (Object item : list) {
out.add(item instanceof Map<?, ?> m ? mask(castMap(m), depth + 1) : item);
}
return out;
}
return value;
}
private boolean isSensitive(String key) {
String lower = key.toLowerCase(Locale.ROOT);
return SENSITIVE_HINTS.stream().anyMatch(lower::contains);
}
/** Blank, the mask placeholder, or any structure that still contains the mask. */
private boolean isRedacted(Object value, int depth) {
if (value == null) {
return true;
}
if (value instanceof String s) {
return s.isBlank() || MASK.equals(s);
}
if (depth >= MAX_DEPTH) {
return false;
}
if (value instanceof Map<?, ?> m) {
return m.values().stream().anyMatch(v -> isRedacted(v, depth + 1));
}
if (value instanceof List<?> list) {
return list.stream().anyMatch(v -> isRedacted(v, depth + 1));
}
return false;
}
@SuppressWarnings("unchecked")
private Map<String, Object> castMap(Map<?, ?> map) {
return (Map<String, Object>) map;
}
}
@@ -0,0 +1,13 @@
package stirling.software.proprietary.access.service;
import stirling.software.proprietary.security.model.User;
/** Resolves whether a user leads a team. */
public interface TeamLeadLookup {
/** Whether the user leads at least one team. */
boolean isAnyTeamLeader(User user);
/** Whether the user leads the given team. */
boolean isLeaderOfTeam(User user, Long teamId);
}
@@ -0,0 +1,116 @@
package stirling.software.proprietary.classification;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
import stirling.software.proprietary.classification.model.TaxonomyValidator;
import stirling.software.proprietary.classification.store.TaxonomyStore;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
/**
* Read/write the caller's team classification taxonomy — the vocabulary the document classifier
* runs against. Team-scoped exactly like policies: every user reads their own team's taxonomy, and
* only a user who may edit policies (a team leader on SaaS, the global admin self-hosted; see
* {@link PolicyManagementAuthority}) may change it. Editing is gated only when login is enabled;
* single-user deployments trust the local operator. A team with no stored taxonomy reads as {@code
* 204} and the classifier falls back to the engine's built-in default.
*/
@RestController
@RequestMapping("/api/v1/classification/taxonomy")
@Hidden
@RequiredArgsConstructor
@Tag(name = "Classification", description = "Team-scoped document-classification taxonomy")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class TaxonomyController {
private final TaxonomyStore taxonomyStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ApplicationProperties applicationProperties;
private final UserServiceInterface userService;
@GetMapping
@Operation(
summary = "Get the team's classification taxonomy",
description =
"Returns the caller's team taxonomy, or 204 when the team has none (the"
+ " classifier then uses the built-in default).")
public ResponseEntity<ClassificationTaxonomy> getTaxonomy() {
return taxonomyStore
.findByTeam(currentTeamId())
.map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.noContent().build());
}
@PutMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Save the team's classification taxonomy",
description =
"Validates and stores the taxonomy for the caller's team, shared by everyone on"
+ " the team. Requires the policy-editor role for the team.")
public ResponseEntity<ClassificationTaxonomy> saveTaxonomy(
@RequestBody ClassificationTaxonomy taxonomy) {
requireEditingAllowed();
try {
TaxonomyValidator.validate(taxonomy);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
ClassificationTaxonomy saved =
taxonomyStore.save(currentTeamId(), taxonomy, currentUsername());
return ResponseEntity.ok(saved);
}
@DeleteMapping
@Operation(
summary = "Reset the team's classification taxonomy",
description =
"Removes the team's stored taxonomy so the classifier falls back to the built-in"
+ " default. Requires the policy-editor role for the team.")
public ResponseEntity<Void> resetTaxonomy() {
requireEditingAllowed();
taxonomyStore.deleteByTeam(currentTeamId());
return ResponseEntity.noContent().build();
}
/**
* Editing the taxonomy requires the editor role for the caller's team — the same gate policies
* use (team leader on SaaS, global admin self-hosted). Single-user deployments (login disabled)
* have no such role, so they trust the local operator.
*/
private void requireEditingAllowed() {
if (!applicationProperties.getSecurity().isEnableLogin()) {
return;
}
if (!policyManagementAuthority.canEditPolicies()) {
throw new ResponseStatusException(
HttpStatus.FORBIDDEN,
"The classification taxonomy may only be changed by a team leader");
}
}
private Long currentTeamId() {
return policyManagementAuthority.currentUserTeamId();
}
private String currentUsername() {
return userService == null ? null : userService.getCurrentUsername();
}
}
@@ -0,0 +1,18 @@
package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* The vocabulary a document is classified against — team-scoped and admin-editable. Its shape
* mirrors the engine's {@code ClassificationTaxonomy} contract (categories owning doc_types, plus
* free-standing cross-cutting tags), so a stored taxonomy is passed to the engine verbatim as the
* per-request override. When a team has no stored taxonomy the engine falls back to its built-in
* default.
*/
public record ClassificationTaxonomy(List<TaxonomyCategory> categories, List<String> tags) {
public ClassificationTaxonomy {
categories = categories == null ? List.of() : List.copyOf(categories);
tags = tags == null ? List.of() : List.copyOf(tags);
}
}
@@ -0,0 +1,15 @@
package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* A structural family of documents, owning the doc_types shaped like it. {@code docTypes} is
* serialized in the engine's camelCase shape (the engine's {@code ClassificationTaxonomy} model
* aliases {@code doc_types} onto it), so a stored taxonomy passes straight through to the engine.
*/
public record TaxonomyCategory(String id, String label, List<TaxonomyDocumentType> docTypes) {
public TaxonomyCategory {
docTypes = docTypes == null ? List.of() : List.copyOf(docTypes);
}
}
@@ -0,0 +1,7 @@
package stirling.software.proprietary.classification.model;
/**
* A specific instrument within a category (e.g. {@code nda} under {@code contract}).
* Category-scoped: the engine enforces that a doc_type can only apply to its owning category.
*/
public record TaxonomyDocumentType(String id, String label) {}
@@ -0,0 +1,87 @@
package stirling.software.proprietary.classification.model;
import java.util.HashSet;
import java.util.Set;
/**
* Structural validation for an admin-supplied (or imported) taxonomy, run before it is stored so a
* malformed vocabulary can never reach the classifier. Mirrors the invariants the engine relies on:
* at least one category, non-blank ids/labels everywhere, ids unique among categories and among the
* doc_types within a category, and non-blank unique tags.
*/
public final class TaxonomyValidator {
private TaxonomyValidator() {}
// Generous upper bounds so a legitimate taxonomy is never blocked, but a single team can't
// store an unbounded blob that would bloat the row, balloon the classifier prompt, or exhaust
// memory on deserialize.
static final int MAX_CATEGORIES = 200;
static final int MAX_DOC_TYPES_PER_CATEGORY = 200;
static final int MAX_TAGS = 500;
static final int MAX_TEXT_LENGTH = 128;
/**
* @throws IllegalArgumentException with a human-readable message when the taxonomy is invalid.
*/
public static void validate(ClassificationTaxonomy taxonomy) {
if (taxonomy == null) {
throw new IllegalArgumentException("Taxonomy is required");
}
if (taxonomy.categories().isEmpty()) {
throw new IllegalArgumentException("Taxonomy must have at least one category");
}
if (taxonomy.categories().size() > MAX_CATEGORIES) {
throw new IllegalArgumentException("Too many categories (max " + MAX_CATEGORIES + ")");
}
if (taxonomy.tags().size() > MAX_TAGS) {
throw new IllegalArgumentException("Too many tags (max " + MAX_TAGS + ")");
}
Set<String> categoryIds = new HashSet<>();
for (TaxonomyCategory category : taxonomy.categories()) {
requireText(category.id(), "Category id");
requireText(category.label(), "Category label");
if (category.docTypes().size() > MAX_DOC_TYPES_PER_CATEGORY) {
throw new IllegalArgumentException(
"Too many sub-categories in '"
+ category.id()
+ "' (max "
+ MAX_DOC_TYPES_PER_CATEGORY
+ ")");
}
if (!categoryIds.add(category.id())) {
throw new IllegalArgumentException("Duplicate category id: " + category.id());
}
Set<String> docTypeIds = new HashSet<>();
for (TaxonomyDocumentType docType : category.docTypes()) {
requireText(docType.id(), "Doc type id");
requireText(docType.label(), "Doc type label");
if (!docTypeIds.add(docType.id())) {
throw new IllegalArgumentException(
"Duplicate doc type id '"
+ docType.id()
+ "' in category '"
+ category.id()
+ "'");
}
}
}
Set<String> tags = new HashSet<>();
for (String tag : taxonomy.tags()) {
requireText(tag, "Tag");
if (!tags.add(tag)) {
throw new IllegalArgumentException("Duplicate tag: " + tag);
}
}
}
private static void requireText(String value, String field) {
if (value == null || value.isBlank()) {
throw new IllegalArgumentException(field + " must not be blank");
}
if (value.length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
field + " is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
}
}
@@ -0,0 +1,37 @@
package stirling.software.proprietary.classification.store;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
/**
* In-memory {@link TaxonomyStore} for tests and any future no-database mode. {@link
* JpaTaxonomyStore} is the runtime bean.
*/
public class InProcessTaxonomyStore implements TaxonomyStore {
private final Map<Long, ClassificationTaxonomy> byTeam = new ConcurrentHashMap<>();
@Override
public Optional<ClassificationTaxonomy> findByTeam(Long teamId) {
return Optional.ofNullable(byTeam.get(key(teamId)));
}
@Override
public ClassificationTaxonomy save(
Long teamId, ClassificationTaxonomy taxonomy, String updatedBy) {
byTeam.put(key(teamId), taxonomy);
return taxonomy;
}
@Override
public boolean deleteByTeam(Long teamId) {
return byTeam.remove(key(teamId)) != null;
}
private static long key(Long teamId) {
return teamId == null ? TaxonomyEntity.NO_TEAM : teamId;
}
}
@@ -0,0 +1,79 @@
package stirling.software.proprietary.classification.store;
import java.time.Instant;
import java.util.Optional;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Durable {@link TaxonomyStore} backed by JPA; the runtime store. Gated on {@code policies.enabled}
* — a team taxonomy only matters when the Classification policy can run — so it shares the policy
* subsystem's on/off switch. The taxonomy is persisted as JSON via {@link TaxonomyEntity}.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaTaxonomyStore implements TaxonomyStore {
private final TaxonomyRepository repository;
private final ObjectMapper objectMapper;
@Override
public Optional<ClassificationTaxonomy> findByTeam(Long teamId) {
Optional<TaxonomyEntity> entity = repository.findById(key(teamId));
if (entity.isEmpty()) {
return Optional.empty();
}
try {
return Optional.of(
objectMapper.readValue(
entity.get().getTaxonomyJson(), ClassificationTaxonomy.class));
} catch (JacksonException e) {
// A stored taxonomy that no longer parses (corruption / manual DB edit) must not break
// classification: drop it so the caller falls back to the built-in default rather than
// surfacing a 500 on every upload for the team.
log.warn(
"Discarding unparseable stored taxonomy for team {}: {}",
teamId,
e.getMessage());
return Optional.empty();
}
}
@Override
public ClassificationTaxonomy save(
Long teamId, ClassificationTaxonomy taxonomy, String updatedBy) {
TaxonomyEntity entity = new TaxonomyEntity();
entity.setTeamId(key(teamId));
entity.setTaxonomyJson(objectMapper.writeValueAsString(taxonomy));
entity.setUpdatedAt(Instant.now());
entity.setUpdatedBy(updatedBy);
repository.save(entity);
return taxonomy;
}
@Override
public boolean deleteByTeam(Long teamId) {
long id = key(teamId);
if (!repository.existsById(id)) {
return false;
}
repository.deleteById(id);
return true;
}
/** Map the nullable team id onto the entity's non-null key (sentinel for the unteamed case). */
private static long key(Long teamId) {
return teamId == null ? TaxonomyEntity.NO_TEAM : teamId;
}
}
@@ -0,0 +1,47 @@
package stirling.software.proprietary.classification.store;
import java.io.Serializable;
import java.time.Instant;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* JPA row for a team's classification taxonomy — one row per team. The taxonomy lives as JSON in
* {@code taxonomyJson} (authoritative on read). {@code teamId} is the natural key; the sentinel
* {@link #NO_TEAM} stands in for the unteamed (login-disabled / self-hosted single-team) case,
* since a primary key can't be null (policies store a nullable {@code team_id}, but this table is
* keyed one-per-team). Kept decoupled from the security entities — {@code teamId} is a plain value,
* not a foreign key — so classification can be enabled or disabled without touching them.
*/
@Entity
@Table(name = "classification_taxonomies")
@NoArgsConstructor
@Getter
@Setter
public class TaxonomyEntity implements Serializable {
private static final long serialVersionUID = 1L;
/** Sentinel key for the unteamed taxonomy (login disabled / no resolvable team). */
public static final long NO_TEAM = 0L;
@Id
@Column(name = "team_id")
private long teamId;
@Column(name = "taxonomy_json", columnDefinition = "text")
private String taxonomyJson;
@Column(name = "updated_at")
private Instant updatedAt;
@Column(name = "updated_by")
private String updatedBy;
}
@@ -0,0 +1,7 @@
package stirling.software.proprietary.classification.store;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface TaxonomyRepository extends JpaRepository<TaxonomyEntity, Long> {}
@@ -0,0 +1,22 @@
package stirling.software.proprietary.classification.store;
import java.util.Optional;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
/**
* Stores one {@link ClassificationTaxonomy} per team. A {@code null} teamId addresses the unteamed
* taxonomy (login disabled / no resolvable team), mirroring how the policy store treats a null
* team.
*/
public interface TaxonomyStore {
/** The team's stored taxonomy, or empty when it has none (callers fall back to the default). */
Optional<ClassificationTaxonomy> findByTeam(Long teamId);
/** Create or replace the team's taxonomy. Returns the stored value. */
ClassificationTaxonomy save(Long teamId, ClassificationTaxonomy taxonomy, String updatedBy);
/** Remove the team's taxonomy (reset to default). Returns whether one existed. */
boolean deleteByTeam(Long teamId);
}
@@ -0,0 +1,193 @@
package stirling.software.proprietary.controller.api;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.classification.store.TaxonomyStore;
import stirling.software.proprietary.model.api.ai.AiPageText;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.PdfContentExtractor;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.node.ObjectNode;
/**
* Dispatchable tool that classifies a PDF and writes the result into its metadata.
*
* <p>Runs as a Classification-policy pipeline step: it reads a bounded page window, asks the AI
* engine to classify the document, and stores the engine's JSON answer — minus the transport-only
* {@code outcome} field — in the custom Info-dictionary key {@link
* PdfMetadataService#CLASSIFICATION_KEY}. Returns the tagged PDF. Not intended for direct client
* use.
*/
@Slf4j
@Hidden
@RestController
@RequestMapping("/api/v1/ai/tools")
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
public class ClassifyTagController {
/** Pages read from each end of the document — mirrors the engine's window. */
private static final int WINDOW_PAGES = 2;
private static final String CLASSIFY_ENDPOINT = "/api/v1/documents/classify";
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private final PdfContentExtractor pdfContentExtractor;
private final PdfMetadataService pdfMetadataService;
private final AiEngineClient aiEngineClient;
private final ObjectMapper objectMapper;
private final UserServiceInterface userService;
/**
* Present only when the policy subsystem is enabled ({@code policies.enabled}); the store and
* team authority are gated on it. Null otherwise, in which case classification falls back to
* the engine's built-in default taxonomy.
*/
private final TaxonomyStore taxonomyStore;
private final PolicyManagementAuthority policyManagementAuthority;
public ClassifyTagController(
CustomPDFDocumentFactory pdfDocumentFactory,
TempFileManager tempFileManager,
PdfContentExtractor pdfContentExtractor,
PdfMetadataService pdfMetadataService,
AiEngineClient aiEngineClient,
ObjectMapper objectMapper,
@Autowired(required = false) UserServiceInterface userService,
@Autowired(required = false) TaxonomyStore taxonomyStore,
@Autowired(required = false) PolicyManagementAuthority policyManagementAuthority) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.tempFileManager = tempFileManager;
this.pdfContentExtractor = pdfContentExtractor;
this.pdfMetadataService = pdfMetadataService;
this.aiEngineClient = aiEngineClient;
this.objectMapper = objectMapper;
this.userService = userService;
this.taxonomyStore = taxonomyStore;
this.policyManagementAuthority = policyManagementAuthority;
}
@PostMapping(value = "/classify-and-tag", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Classify a PDF and tag its metadata",
description =
"Reads the first two and last two pages, classifies the document via the AI"
+ " engine, and stores the result in the StirlingPDFClassification"
+ " metadata field. Dispatched by the Classification policy; not"
+ " intended for direct client use.")
public ResponseEntity<Resource> classifyAndTag(
@RequestParam("fileInput") MultipartFile fileInput) throws IOException {
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
String fileName = safeFileName(fileInput.getOriginalFilename());
List<AiPageText> pages = extractWindow(document);
String requestBody =
objectMapper.writeValueAsString(
new ClassifyEngineRequest(fileName, pages, resolveTaxonomyOverride()));
String userId = userService != null ? userService.getCurrentUsername() : null;
String responseJson = aiEngineClient.post(CLASSIFY_ENDPOINT, requestBody, userId);
pdfMetadataService.setClassificationMetadata(document, toMetadataValue(responseJson));
log.debug("[classify-and-tag] tagged {} ({} window pages)", fileName, pages.size());
return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
}
}
private List<AiPageText> extractWindow(PDDocument document) throws IOException {
List<AiPageText> pages = new ArrayList<>();
for (int pageNumber : windowPageNumbers(document.getNumberOfPages(), WINDOW_PAGES)) {
String text = pdfContentExtractor.extractPageTextRaw(document, pageNumber);
if (text != null && !text.isBlank()) {
pages.add(new AiPageText(pageNumber, text));
}
}
return pages;
}
/** First and last {@code window} page numbers (1-based), de-duplicated and in order. */
static List<Integer> windowPageNumbers(int pageCount, int window) {
Set<Integer> numbers = new LinkedHashSet<>();
for (int page = 1; page <= Math.min(window, pageCount); page++) {
numbers.add(page);
}
for (int page = Math.max(1, pageCount - window + 1); page <= pageCount; page++) {
numbers.add(page);
}
return new ArrayList<>(numbers);
}
/** Drop the transport-only {@code outcome} discriminator; keep the rest verbatim. */
private String toMetadataValue(String engineResponseJson) {
JsonNode node = objectMapper.readTree(engineResponseJson);
if (node instanceof ObjectNode object) {
object.remove("outcome");
}
return objectMapper.writeValueAsString(node);
}
private static String safeFileName(String originalFilename) {
String name = Filenames.toSimpleFileName(originalFilename);
return (name == null || name.isBlank()) ? "classified.pdf" : name;
}
/**
* Resolve the caller's team taxonomy and return it in the engine's shape to classify against;
* {@code null} falls back to the engine's generated default. The stored taxonomy is already in
* the engine's camelCase shape ({@code categories}/{@code docTypes}/{@code tags}), so it is
* passed through verbatim. Returns null when the policy subsystem is disabled (no store), when
* the team has no stored taxonomy, or when the team can't be resolved.
*/
private JsonNode resolveTaxonomyOverride() {
if (taxonomyStore == null) {
return null;
}
Long teamId =
policyManagementAuthority == null
? null
: policyManagementAuthority.currentUserTeamId();
return taxonomyStore
.findByTeam(teamId)
.map(taxonomy -> (JsonNode) objectMapper.valueToTree(taxonomy))
.orElse(null);
}
/** Request body for the engine's {@code /api/v1/documents/classify} endpoint. */
@JsonInclude(JsonInclude.Include.NON_NULL)
private record ClassifyEngineRequest(
String fileName, List<AiPageText> pages, JsonNode taxonomy) {}
}
@@ -0,0 +1,81 @@
package stirling.software.proprietary.integration.controller;
import java.util.List;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.integration.dto.IntegrationConfigRequest;
import stirling.software.proprietary.integration.dto.IntegrationConfigResponse;
import stirling.software.proprietary.integration.service.IntegrationConfigService;
import stirling.software.proprietary.security.model.User;
/** CRUD for S3/MCP/API integration configs. Secrets are never returned. */
@RestController
@RequestMapping("/api/v1/integrations")
@RequiredArgsConstructor
@PreAuthorize("isAuthenticated()")
@Tag(name = "Integrations", description = "Manage S3/MCP/API integration configurations")
public class IntegrationConfigController {
private final IntegrationConfigService service;
@GetMapping
public ResponseEntity<List<IntegrationConfigResponse>> list(
@AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(
service.listVisible(user).stream().map(c -> service.toResponse(c, user)).toList());
}
@PostMapping
public ResponseEntity<IntegrationConfigResponse> create(
@RequestBody IntegrationConfigRequest request, @AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.create(request, user), user));
}
@GetMapping("/{id}")
public ResponseEntity<IntegrationConfigResponse> get(
@PathVariable Long id, @AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.getForUse(id, user), user));
}
@PutMapping("/{id}")
public ResponseEntity<IntegrationConfigResponse> update(
@PathVariable Long id,
@RequestBody IntegrationConfigRequest request,
@AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.update(id, request, user), user));
}
@DeleteMapping("/{id}")
public ResponseEntity<Void> delete(@PathVariable Long id, @AuthenticationPrincipal User user) {
requireUser(user);
service.delete(id, user);
return ResponseEntity.noContent().build();
}
private void requireUser(User user) {
if (user == null) {
throw new ResponseStatusException(HttpStatus.UNAUTHORIZED, "Authentication required");
}
}
}
@@ -0,0 +1,135 @@
package stirling.software.proprietary.integration.crypto;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.GeneralSecurityException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import jakarta.annotation.PostConstruct;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.InstallationPathConfig;
/**
* AES-256-GCM for stored credentials. Key from property, env var, or an auto-generated key file.
*/
@Component
@Slf4j
public class CredentialEncryption {
private static final String ALGORITHM = "AES";
private static final String TRANSFORMATION = "AES/GCM/NoPadding";
private static final int GCM_TAG_BITS = 128;
private static final int IV_BYTES = 12;
private static final String KEY_FILE = "credential-encryption.key";
private static final SecureRandom RANDOM = new SecureRandom();
private static volatile SecretKey key;
private final String configuredKey;
public CredentialEncryption(
@Value("${stirling.security.credentialEncryptionKey:}") String configuredKey) {
this.configuredKey = configuredKey;
}
@PostConstruct
void init() {
key = resolveKey();
log.info("Credential encryption initialised (AES-256-GCM)");
}
private SecretKey resolveKey() {
String configured = configuredKey;
if (configured == null || configured.isBlank()) {
configured = System.getenv("STIRLING_CREDENTIAL_ENCRYPTION_KEY");
}
if (configured != null && !configured.isBlank()) {
return new SecretKeySpec(Base64.getDecoder().decode(configured.trim()), ALGORITHM);
}
return loadOrCreateKeyFile();
}
private SecretKey loadOrCreateKeyFile() {
Path path = Path.of(InstallationPathConfig.getConfigPath(), KEY_FILE);
try {
if (Files.exists(path)) {
String encoded = Files.readString(path).trim();
return new SecretKeySpec(Base64.getDecoder().decode(encoded), ALGORITHM);
}
KeyGenerator generator = KeyGenerator.getInstance(ALGORITHM);
generator.init(256);
SecretKey generated = generator.generateKey();
Files.createDirectories(path.getParent());
Files.writeString(path, Base64.getEncoder().encodeToString(generated.getEncoded()));
log.warn(
"Generated a new credential encryption key at {}. Back this file up: losing it"
+ " makes stored integration secrets unrecoverable.",
path);
return generated;
} catch (Exception e) {
throw new IllegalStateException("Unable to initialise credential encryption key", e);
}
}
public static String encrypt(String plaintext) {
if (plaintext == null) {
return null;
}
try {
byte[] iv = new byte[IV_BYTES];
RANDOM.nextBytes(iv);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.ENCRYPT_MODE, requireKey(), new GCMParameterSpec(GCM_TAG_BITS, iv));
byte[] ciphertext = cipher.doFinal(plaintext.getBytes(StandardCharsets.UTF_8));
byte[] combined = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, combined, 0, iv.length);
System.arraycopy(ciphertext, 0, combined, iv.length, ciphertext.length);
return Base64.getEncoder().encodeToString(combined);
} catch (GeneralSecurityException e) {
throw new IllegalStateException("Failed to encrypt credential", e);
}
}
public static String decrypt(String stored) {
if (stored == null) {
return null;
}
try {
byte[] combined = Base64.getDecoder().decode(stored);
byte[] iv = Arrays.copyOfRange(combined, 0, IV_BYTES);
byte[] ciphertext = Arrays.copyOfRange(combined, IV_BYTES, combined.length);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.DECRYPT_MODE, requireKey(), new GCMParameterSpec(GCM_TAG_BITS, iv));
return new String(cipher.doFinal(ciphertext), StandardCharsets.UTF_8);
} catch (GeneralSecurityException e) {
throw new IllegalStateException("Failed to decrypt credential", e);
}
}
private static SecretKey requireKey() {
SecretKey current = key;
if (current == null) {
throw new IllegalStateException("Credential encryption not initialised");
}
return current;
}
/** For tests. */
static void initialiseForTesting(SecretKey testKey) {
key = testKey;
}
}
@@ -0,0 +1,19 @@
package stirling.software.proprietary.integration.crypto;
import jakarta.persistence.AttributeConverter;
import jakarta.persistence.Converter;
/** Transparently encrypts/decrypts a string column at rest via {@link CredentialEncryption}. */
@Converter
public class EncryptedStringConverter implements AttributeConverter<String, String> {
@Override
public String convertToDatabaseColumn(String attribute) {
return CredentialEncryption.encrypt(attribute);
}
@Override
public String convertToEntityAttribute(String dbData) {
return CredentialEncryption.decrypt(dbData);
}
}
@@ -0,0 +1,18 @@
package stirling.software.proprietary.integration.dto;
import java.util.Map;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationType;
/** Create/update payload for an integration config. Sensitive config values left blank are kept. */
public record IntegrationConfigRequest(
IntegrationType integrationType,
String name,
OwnerScope scope,
Long ownerTeamId,
Boolean enabled,
Boolean locked,
DefaultAccessPolicy defaultAccess,
Map<String, Object> config) {}
@@ -0,0 +1,24 @@
package stirling.software.proprietary.integration.dto;
import java.time.LocalDateTime;
import java.util.Map;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationType;
/** Integration config view. Sensitive config values are masked. */
public record IntegrationConfigResponse(
Long id,
IntegrationType integrationType,
String name,
OwnerScope scope,
Long ownerUserId,
Long ownerTeamId,
boolean enabled,
boolean locked,
DefaultAccessPolicy defaultAccess,
Map<String, Object> config,
boolean canManage,
LocalDateTime createdAt,
LocalDateTime updatedAt) {}
@@ -0,0 +1,67 @@
package stirling.software.proprietary.integration.model;
import java.io.Serializable;
import java.time.LocalDateTime;
import org.hibernate.annotations.CreationTimestamp;
import org.hibernate.annotations.UpdateTimestamp;
import jakarta.persistence.Column;
import jakarta.persistence.Convert;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Index;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.integration.crypto.EncryptedStringConverter;
/** A named S3/MCP/API integration config; scope and ownership live on {@link OwnedResource}. */
@Entity
@Table(
name = "integration_configs",
indexes = {
@Index(name = "idx_integration_configs_owner", columnList = "owner_user_id"),
@Index(name = "idx_integration_configs_type", columnList = "integration_type"),
@Index(name = "idx_integration_configs_scope", columnList = "scope")
})
@NoArgsConstructor
@Getter
@Setter
public class IntegrationConfig extends OwnedResource implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "integration_config_id")
private Long id;
@Enumerated(EnumType.STRING)
@Column(name = "integration_type", nullable = false, length = 32)
private IntegrationType integrationType;
@Column(name = "name", nullable = false)
private String name;
// Type-specific fields as an AES-GCM encrypted JSON blob.
@Convert(converter = EncryptedStringConverter.class)
@Column(name = "config_encrypted", columnDefinition = "text")
private String config;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
@UpdateTimestamp
@Column(name = "updated_at")
private LocalDateTime updatedAt;
}
@@ -0,0 +1,8 @@
package stirling.software.proprietary.integration.model;
/** Kind of external integration a stored config describes. */
public enum IntegrationType {
S3,
MCP,
API
}
@@ -0,0 +1,21 @@
package stirling.software.proprietary.integration.repository;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
@Repository
public interface IntegrationConfigRepository extends JpaRepository<IntegrationConfig, Long> {
List<IntegrationConfig> findByOwnerUser(User ownerUser);
List<IntegrationConfig> findByOwnerTeam(Team ownerTeam);
List<IntegrationConfig> findByScope(OwnerScope scope);
}
@@ -0,0 +1,226 @@
package stirling.software.proprietary.integration.service;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.server.ResponseStatusException;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.access.service.SecretMasker;
import stirling.software.proprietary.integration.dto.IntegrationConfigRequest;
import stirling.software.proprietary.integration.dto.IntegrationConfigResponse;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.ObjectMapper;
/** CRUD for {@link IntegrationConfig}; delegates ownership and masking to shared services. */
@Service
@RequiredArgsConstructor
@Slf4j
@Transactional(readOnly = true)
public class IntegrationConfigService {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private final IntegrationConfigRepository repository;
private final OwnershipService ownership;
private final SecretMasker secretMasker;
// ---- commands ----
@Transactional
public IntegrationConfig create(IntegrationConfigRequest request, User currentUser) {
OwnerScope scope = request.scope() == null ? OwnerScope.USER : request.scope();
IntegrationConfig cfg = new IntegrationConfig();
cfg.setIntegrationType(require(request.integrationType(), "integrationType"));
cfg.setName(require(request.name(), "name"));
cfg.setEnabled(request.enabled() == null || request.enabled());
cfg.setLocked(request.locked() != null && request.locked());
cfg.setDefaultAccess(
request.defaultAccess() == null
? DefaultAccessPolicy.EXPLICIT_ONLY
: request.defaultAccess());
ownership.assignOwnership(
cfg,
scope,
request.ownerTeamId(),
currentUser,
() -> lockedServerExists(cfg.getIntegrationType()));
cfg.setConfig(writeJson(secretMasker.sanitize(request.config())));
return repository.save(cfg);
}
@Transactional
public IntegrationConfig update(Long id, IntegrationConfigRequest request, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canManage(TYPE, cfg, currentUser)) {
throw forbidden("You cannot manage this integration");
}
if (cfg.isLocked() && !ownership.isAdmin(currentUser)) {
throw forbidden("This integration is locked by an administrator");
}
if (request.name() != null) {
cfg.setName(request.name());
}
if (request.enabled() != null) {
cfg.setEnabled(request.enabled());
}
if (request.locked() != null && request.locked() != cfg.isLocked()) {
if (!ownership.isAdmin(currentUser)) {
throw forbidden("Only administrators can change the locked flag");
}
cfg.setLocked(request.locked());
}
if (request.defaultAccess() != null) {
cfg.setDefaultAccess(request.defaultAccess());
}
if (request.config() != null) {
cfg.setConfig(
writeJson(secretMasker.merge(readJson(cfg.getConfig()), request.config())));
}
return repository.save(cfg);
}
@Transactional
public void delete(Long id, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canManage(TYPE, cfg, currentUser)) {
throw forbidden("You cannot manage this integration");
}
repository.delete(cfg);
}
// ---- queries ----
public IntegrationConfig getForUse(Long id, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canUse(TYPE, cfg, currentUser)) {
throw forbidden("You cannot access this integration");
}
return cfg;
}
/**
* All configs the user owns, plus server/team configs and grant-shared configs they may use.
*/
public List<IntegrationConfig> listVisible(User currentUser) {
Map<Long, IntegrationConfig> byId = new LinkedHashMap<>();
for (IntegrationConfig c : repository.findByOwnerUser(currentUser)) {
byId.put(c.getId(), c);
}
for (IntegrationConfig c : repository.findByScope(OwnerScope.SERVER)) {
if (ownership.canUse(TYPE, c, currentUser)) {
byId.put(c.getId(), c);
}
}
if (currentUser.getTeam() != null) {
for (IntegrationConfig c : repository.findByOwnerTeam(currentUser.getTeam())) {
if (ownership.canUse(TYPE, c, currentUser)) {
byId.put(c.getId(), c);
}
}
}
for (String rid : ownership.grantedResourceIds(TYPE, currentUser)) {
if (rid == null || rid.isBlank()) {
continue;
}
Long cid;
try {
cid = Long.valueOf(rid);
} catch (NumberFormatException e) {
continue;
}
if (byId.containsKey(cid)) {
continue;
}
repository
.findById(cid)
.filter(c -> ownership.canUse(TYPE, c, currentUser))
.ifPresent(c -> byId.put(c.getId(), c));
}
return new ArrayList<>(byId.values());
}
public IntegrationConfigResponse toResponse(IntegrationConfig cfg, User user) {
return new IntegrationConfigResponse(
cfg.getId(),
cfg.getIntegrationType(),
cfg.getName(),
cfg.getScope(),
cfg.getOwnerUserId(),
cfg.getOwnerTeamId(),
cfg.isEnabled(),
cfg.isLocked(),
cfg.getDefaultAccess(),
secretMasker.mask(readJson(cfg.getConfig())),
ownership.canManage(TYPE, cfg, user),
cfg.getCreatedAt(),
cfg.getUpdatedAt());
}
// ---- integration-specific glue ----
/** A non-admin can't create a personal config of a type an admin has locked at server scope. */
private boolean lockedServerExists(IntegrationType type) {
return repository.findByScope(OwnerScope.SERVER).stream()
.anyMatch(c -> c.getIntegrationType() == type && c.isLocked());
}
private IntegrationConfig load(Long id) {
return repository
.findById(id)
.orElseThrow(
() ->
new ResponseStatusException(
HttpStatus.NOT_FOUND, "Integration not found"));
}
private String writeJson(Map<String, Object> config) {
try {
return OBJECT_MAPPER.writeValueAsString(config == null ? Map.of() : config);
} catch (Exception e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Invalid config payload");
}
}
private Map<String, Object> readJson(String json) {
if (json == null || json.isBlank()) {
return new LinkedHashMap<>();
}
try {
return OBJECT_MAPPER.readValue(
json, new TypeReference<LinkedHashMap<String, Object>>() {});
} catch (Exception e) {
log.error("Failed to parse integration config JSON", e);
return new LinkedHashMap<>();
}
}
private <T> T require(T value, String field) {
if (value == null || (value instanceof String s && s.isBlank())) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, field + " is required");
}
return value;
}
private ResponseStatusException forbidden(String message) {
return new ResponseStatusException(HttpStatus.FORBIDDEN, message);
}
}
@@ -21,4 +21,11 @@ public class AiWorkflowResultFile {
@Schema(description = "MIME type of the file", example = "application/pdf")
private String contentType;
@Schema(
description =
"Index into the request's fileInputs that this output was derived from, or null"
+ " when it has no single source (e.g. a merge, or a generated file)."
+ " Lets the client replace that input in place as a new version.")
private Integer sourceIndex;
}
@@ -2,6 +2,7 @@ package stirling.software.proprietary.policy.controller;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
@@ -52,11 +53,15 @@ import stirling.software.proprietary.policy.model.PolicyInputs;
import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.model.PolicyRunStatus;
import stirling.software.proprietary.policy.model.PolicyRunView;
import stirling.software.proprietary.policy.overview.PoliciesOverviewResponse;
import stirling.software.proprietary.policy.overview.PolicyOverviewService;
import stirling.software.proprietary.policy.progress.PolicyProgressListener;
import stirling.software.proprietary.policy.source.SourceAccessGuard;
import stirling.software.proprietary.policy.source.SourceStore;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.policy.trigger.PolicyTrigger;
import stirling.software.proprietary.policy.trigger.PolicyTriggerManager;
import stirling.software.proprietary.policy.trigger.TriggerInfo;
/**
* Policy CRUD plus pipeline runs (stored or ad-hoc). Runs are async: returns a run id, poll {@code
@@ -80,6 +85,8 @@ public class PolicyController {
private final PolicyAccessGuard policyAccessGuard;
private final PolicyManagementAuthority policyManagementAuthority;
private final PolicyTriggerManager policyTriggerManager;
private final PolicyOverviewService policyOverviewService;
private final List<PolicyTrigger> policyTriggers;
private final ApplicationProperties applicationProperties;
private final TempFileManager tempFileManager;
private final JobOwnershipService jobOwnershipService;
@@ -287,6 +294,31 @@ public class PolicyController {
return policyAccessGuard.visibleFrom(policyStore);
}
@GetMapping("/overview")
@Operation(
summary = "Pipelines overview",
description =
"Returns the KPI strip plus one row per policy the caller's team owns, each with"
+ " its referenced sources resolved to names, its pipeline steps, and a"
+ " trigger/output summary. Backs the portal's all-pipelines surface.")
public PoliciesOverviewResponse overview() {
return policyOverviewService.overview();
}
@GetMapping("/triggers")
@Operation(
summary = "List available triggers",
description =
"Lists each trigger kind with whether it needs a source and which source types"
+ " it supports, so the UI can offer triggers and pair them with the"
+ " right sources.")
public List<TriggerInfo> triggers() {
return policyTriggers.stream()
.map(TriggerInfo::of)
.sorted(Comparator.comparing(TriggerInfo::type))
.toList();
}
@GetMapping("/{policyId}")
@Operation(summary = "Get a policy by id")
public ResponseEntity<Policy> getPolicy(@PathVariable String policyId) {
@@ -337,6 +369,26 @@ public class PolicyController {
return ResponseEntity.accepted().body(new JobResponse<>(true, runId, null));
}
@PostMapping("/{policyId}/trigger")
@Operation(
summary = "Run a stored policy against its sources",
description =
"Pulls the policy's configured sources and runs the pipeline now, regardless of"
+ " the enabled flag (which only gates automatic triggering). Returns"
+ " the ids of the runs started; poll the run-status endpoint for each."
+ " Empty when the sources yielded no work to do.")
public ResponseEntity<List<String>> trigger(@PathVariable String policyId) {
Policy policy =
policyStore
.get(policyId)
.filter(policyAccessGuard::canAccess)
.orElseThrow(
() ->
new ResponseStatusException(
HttpStatus.NOT_FOUND, "No policy: " + policyId));
return ResponseEntity.accepted().body(policyRunner.run(policy));
}
private static void requireRunnable(PipelineDefinition definition) {
if (definition.steps().isEmpty()) {
throw new ResponseStatusException(
@@ -8,7 +8,11 @@ import tools.jackson.databind.JsonNode;
/**
* Result of a {@link PolicyExecutor} run. {@code files} are final temp files (not yet stored).
* {@code report}/{@code reportTool} carry the last step's structured report and its operation, or
* null if no step produced one.
* {@code origins} is parallel to {@code files}: each entry is the index into the original pipeline
* inputs that the output traces back to, or {@code null} when it has no single source (e.g. a merge
* combining several inputs, or a generated file). Callers use it to map an output back onto the
* file it came from. {@code report}/{@code reportTool} carry the last step's structured report and
* its operation, or null if no step produced one.
*/
public record PolicyExecutionResult(List<Resource> files, JsonNode report, String reportTool) {}
public record PolicyExecutionResult(
List<Resource> files, List<Integer> origins, JsonNode report, String reportTool) {}
@@ -58,6 +58,11 @@ public class PolicyExecutor {
// payload the tool surfaced alongside or instead of a file.
private record ToolResult(List<Resource> files, JsonNode report) {}
// A step's output files paired with each file's origin (the index into the original pipeline
// inputs it traces back to, or null when it has no single source). Origins compose across steps
// so the final result can be mapped back onto the files that entered the pipeline.
private record StepOutput(List<Resource> files, List<Integer> origins, JsonNode report) {}
/**
* Run every step in order, feeding each step's output into the next. Supporting files in {@code
* inputs} bind to named file fields and never enter the document stream.
@@ -75,6 +80,12 @@ public class PolicyExecutor {
List<Resource> currentFiles = inputs.primary();
Map<String, List<Resource>> supportingFiles = inputs.supportingFiles();
// Seed each input with its own index as origin; steps carry these through so the final
// outputs can be traced back to the files that entered the pipeline.
List<Integer> currentOrigins = new ArrayList<>();
for (int k = 0; k < currentFiles.size(); k++) {
currentOrigins.add(k);
}
// Last non-null report wins: the terminal step defines the output.
JsonNode lastReport = null;
String lastReportTool = null;
@@ -87,8 +98,10 @@ public class PolicyExecutor {
"Pipeline step " + (i + 1) + " has no operation");
}
listener.onStepStart(i + 1, steps.size(), operation);
ToolResult stepResult = executeStep(step, currentFiles, supportingFiles);
StepOutput stepResult =
executeStep(step, currentFiles, currentOrigins, supportingFiles);
currentFiles = stepResult.files();
currentOrigins = stepResult.origins();
if (stepResult.report() != null) {
lastReport = stepResult.report();
lastReportTool = operation;
@@ -96,7 +109,7 @@ public class PolicyExecutor {
listener.onStepComplete(i + 1, steps.size(), operation);
}
return new PolicyExecutionResult(currentFiles, lastReport, lastReportTool);
return new PolicyExecutionResult(currentFiles, currentOrigins, lastReport, lastReportTool);
}
/**
@@ -104,32 +117,50 @@ public class PolicyExecutor {
* responses are unpacked so each inner file is its own result (e.g. split). For per-file
* dispatch the first non-null report wins.
*/
private ToolResult executeStep(
private StepOutput executeStep(
PipelineStep step,
List<Resource> inputFiles,
List<Integer> inputOrigins,
Map<String, List<Resource>> supportingFiles)
throws IOException {
requireAcceptedTypes(step.operation(), inputFiles);
List<Resource> files = new ArrayList<>();
List<Integer> origins = new ArrayList<>();
JsonNode report = null;
if (toolMetadataService.isMultiInput(step.operation())) {
// One call over all inputs. The outputs derive from a single input only when exactly
// one entered; otherwise (a genuine merge) there is no single source.
ToolResult r = callEndpoint(step, inputFiles, supportingFiles);
files.addAll(r.files());
Integer origin = inputOrigins.size() == 1 ? inputOrigins.get(0) : null;
for (Resource file : r.files()) {
files.add(file);
origins.add(origin);
}
report = r.report();
} else if (inputFiles.isEmpty()) {
ToolResult r = callEndpoint(step, List.of(), supportingFiles);
files.addAll(r.files());
for (Resource file : r.files()) {
files.add(file);
origins.add(null);
}
report = r.report();
} else {
for (Resource file : inputFiles) {
ToolResult r = callEndpoint(step, List.of(file), supportingFiles);
files.addAll(r.files());
// One call per file: every output of this call inherits that input's origin, so a 1:1
// op keeps its chain and a split (one input, many outputs) tags each output with the
// same source.
for (int k = 0; k < inputFiles.size(); k++) {
Integer origin = inputOrigins.get(k);
ToolResult r = callEndpoint(step, List.of(inputFiles.get(k)), supportingFiles);
for (Resource file : r.files()) {
files.add(file);
origins.add(origin);
}
if (report == null) {
report = r.report();
}
}
}
return new ToolResult(files, report);
return new StepOutput(files, origins, report);
}
/**
@@ -1,6 +1,7 @@
package stirling.software.proprietary.policy.engine;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;
@@ -20,6 +21,7 @@ import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.model.PolicyRunStatus;
import stirling.software.proprietary.policy.progress.PolicyProgressListener;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceDocCounter;
import stirling.software.proprietary.policy.source.SourceStore;
/**
@@ -36,19 +38,21 @@ public class PolicyRunner {
private final PolicyEngine policyEngine;
private final List<InputSource> inputSources;
private final SourceStore sourceStore;
private final SourceDocCounter docCounter;
/**
* Trigger entry point. Pulls every referenced source; each yielded unit becomes its own run so
* one failure does not affect the others. No sources means one run with no input (generator
* pipeline). Missing or disabled sources are skipped so one broken reference does not stop the
* rest.
* rest. Returns the ids of the runs it started (empty when sources yielded no work), so a
* manual trigger can report back which runs to follow.
*/
public void run(Policy policy) {
public List<String> run(Policy policy) {
List<String> sourceIds = policy.sourceIds();
if (sourceIds.isEmpty()) {
startRun(policy, PolicyInputs.of(List.of()), unused -> {});
return;
return List.of(startRun(policy, PolicyInputs.of(List.of()), unused -> {}));
}
List<String> runIds = new ArrayList<>();
for (String sourceId : sourceIds) {
Source source = sourceStore.get(sourceId).orElse(null);
if (source == null) {
@@ -63,8 +67,9 @@ public class PolicyRunner {
policy.id());
continue;
}
pullAndRun(policy, source.toInputSpec());
runIds.addAll(pullAndRun(policy, sourceId, source.toInputSpec()));
}
return runIds;
}
/** Run a stored policy on caller-supplied files (e.g. manual upload), bypassing its sources. */
@@ -79,14 +84,18 @@ public class PolicyRunner {
return policyEngine.submit(definition, inputs, listener);
}
private void pullAndRun(Policy policy, InputSpec spec) {
/**
* Resolves the source and starts a run per unit; records how many documents the source fed and
* returns the ids of the runs started.
*/
private List<String> pullAndRun(Policy policy, String sourceId, InputSpec spec) {
InputSource source = sourceFor(spec);
if (source == null) {
log.warn(
"No input source for type '{}' (policy {}); skipping",
spec.type(),
policy.id());
return;
return List.of();
}
List<ResolvedInput> work;
try {
@@ -97,19 +106,25 @@ public class PolicyRunner {
spec.type(),
policy.id(),
e.getMessage());
return;
return List.of();
}
List<String> runIds = new ArrayList<>();
long docsFed = 0;
for (ResolvedInput unit : work) {
startRun(policy, unit.inputs(), unit.onComplete());
runIds.add(startRun(policy, unit.inputs(), unit.onComplete()));
docsFed += unit.inputs().primary().size();
}
docCounter.record(sourceId, docsFed);
return runIds;
}
private void startRun(Policy policy, PolicyInputs inputs, Consumer<Boolean> onComplete) {
private String startRun(Policy policy, PolicyInputs inputs, Consumer<Boolean> onComplete) {
log.info("Running policy {} ({})", policy.id(), policy.name());
PolicyRunHandle handle =
policyEngine.runPolicy(policy, inputs, PolicyProgressListener.NOOP);
handle.completion()
.whenComplete((run, throwable) -> onComplete.accept(succeeded(run, throwable)));
return handle.runId();
}
private static boolean succeeded(PolicyRun run, Throwable throwable) {
@@ -0,0 +1,6 @@
package stirling.software.proprietary.policy.overview;
import java.util.List;
/** The Pipelines overview payload: a KPI strip plus one row per policy. */
public record PoliciesOverviewResponse(List<PolicyKpi> kpis, List<PolicyView> pipelines) {}
@@ -0,0 +1,4 @@
package stirling.software.proprietary.policy.overview;
/** One headline figure in the Pipelines overview strip. */
public record PolicyKpi(long value, String description) {}
@@ -0,0 +1,102 @@
package stirling.software.proprietary.policy.overview;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.policy.config.PolicyAccessGuard;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.model.TriggerConfig;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceAccessGuard;
import stirling.software.proprietary.policy.source.SourceStore;
import stirling.software.proprietary.policy.store.PolicyStore;
/**
* Builds the Pipelines overview: every policy the caller's team owns, each annotated with its
* referenced sources (resolved to display names), its pipeline steps, and a trigger/output summary.
* Source names are resolved from the team's sources in memory rather than persisted on the policy,
* so the view always reflects the live source set. This is the "all pipelines" admin surface; the
* user-facing Policies page builds only a friendly subset of the same backend policies.
*/
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyOverviewService {
private final PolicyStore policyStore;
private final SourceStore sourceStore;
private final PolicyAccessGuard policyAccessGuard;
private final SourceAccessGuard sourceAccessGuard;
public PoliciesOverviewResponse overview() {
List<Policy> policies = policyAccessGuard.visibleFrom(policyStore);
Map<String, String> sourceNames = sourceNames();
List<PolicyView> views =
policies.stream()
.map(policy -> toView(policy, sourceNames))
.sorted(
Comparator.comparing(
PolicyView::name, String.CASE_INSENSITIVE_ORDER))
.toList();
return new PoliciesOverviewResponse(buildKpis(policies), views);
}
/** Display names for every source the caller's team can see, keyed by source id. */
private Map<String, String> sourceNames() {
Map<String, String> names = new HashMap<>();
for (Source source : sourceAccessGuard.visibleFrom(sourceStore)) {
names.put(source.id(), source.name());
}
return names;
}
private static PolicyView toView(Policy policy, Map<String, String> sourceNames) {
List<PolicyView.SourceRef> sources =
policy.sourceIds().stream()
// An unresolved id (source deleted, or not visible) falls back to the id so
// the row still renders rather than dropping the reference silently.
.map(id -> new PolicyView.SourceRef(id, sourceNames.getOrDefault(id, id)))
.toList();
List<String> steps = policy.steps().stream().map(PipelineStep::operation).toList();
return new PolicyView(
policy.id(),
policy.name(),
policy.enabled(),
policy.enabled() ? "active" : "paused",
triggerSummary(policy.trigger()),
sources,
steps,
outputSummary(policy.output()),
policy.owner());
}
/** A null trigger is a manual-only policy; otherwise the trigger's type keys the summary. */
private static String triggerSummary(TriggerConfig trigger) {
return trigger == null ? "manual" : trigger.type();
}
private static String outputSummary(OutputSpec output) {
return output == null ? "inline" : output.type();
}
private static List<PolicyKpi> buildKpis(List<Policy> policies) {
long total = policies.size();
long active = policies.stream().filter(Policy::enabled).count();
long paused = total - active;
return List.of(
new PolicyKpi(total, "pipelines"),
new PolicyKpi(active, "running automatically"),
new PolicyKpi(paused, "paused"));
}
}
@@ -0,0 +1,24 @@
package stirling.software.proprietary.policy.overview;
import java.util.List;
/**
* One row in the Pipelines overview: a stored policy shown for the admin portal, with its
* referenced sources resolved to names and its pipeline summarised. The portal's "all pipelines"
* surface lists every backend policy (the user-facing Policies page builds only a friendly subset
* of these).
*/
public record PolicyView(
String id,
String name,
boolean enabled,
String status,
String trigger,
List<SourceRef> sources,
List<String> steps,
String output,
String owner) {
/** A source a policy pulls documents from, resolved to its display name. */
public record SourceRef(String id, String name) {}
}
@@ -0,0 +1,15 @@
package stirling.software.proprietary.policy.source;
/**
* Per-source document throughput for the overview row: how many documents a source has fed into
* runs in total and over the trailing 24-hour and 30-day windows. Counts documents <em>fed</em>
* (picked up by a run), so a snapshot-mode source that re-reads the same files each run counts them
* per run.
*/
public record DocStats(long total, long last24h, long last30d) {
/** Number of trailing daily buckets in a source's daily series. */
public static final int DAYS = 30;
public static final DocStats ZERO = new DocStats(0, 0, 0);
}
@@ -0,0 +1,64 @@
package stirling.software.proprietary.policy.source;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Supplier;
/**
* In-memory {@link SourceDocCounter} for tests and any future no-database mode. Holds hourly
* buckets per source; the clock is injectable so window boundaries can be exercised
* deterministically. {@link JpaSourceDocCounter} is the runtime bean.
*/
public class InProcessSourceDocCounter implements SourceDocCounter {
private final Supplier<Instant> clock;
private final Map<String, Map<Long, Long>> bucketsBySource = new ConcurrentHashMap<>();
public InProcessSourceDocCounter() {
this(Instant::now);
}
public InProcessSourceDocCounter(Supplier<Instant> clock) {
this.clock = clock;
}
@Override
public void record(String sourceId, long docs) {
if (docs <= 0) {
return;
}
bucketsBySource
.computeIfAbsent(sourceId, key -> new ConcurrentHashMap<>())
.merge(currentHour(), docs, Long::sum);
}
@Override
public Map<String, DocStats> statsFor(Collection<String> sourceIds) {
long now = currentHour();
Map<String, DocStats> stats = new HashMap<>();
for (String id : sourceIds) {
Map<Long, Long> buckets = bucketsBySource.getOrDefault(id, Map.of());
long total = buckets.values().stream().mapToLong(Long::longValue).sum();
long last24h =
SourceDocWindows.sumSince(buckets, now - (SourceDocWindows.HOURS_IN_24H - 1));
long last30d = SourceDocWindows.sumSince(buckets, SourceDocWindows.firstDayHour(now));
stats.put(id, new DocStats(total, last24h, last30d));
}
return stats;
}
@Override
public List<Long> dailySeriesFor(String sourceId) {
long now = currentHour();
Map<Long, Long> buckets = bucketsBySource.getOrDefault(sourceId, Map.of());
return SourceDocWindows.series(SourceDocWindows.byDay(buckets), now / 24);
}
private long currentHour() {
return clock.get().getEpochSecond() / 3600;
}
}
@@ -0,0 +1,154 @@
package stirling.software.proprietary.policy.source;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.function.IntSupplier;
import java.util.function.Supplier;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
/**
* Durable {@link SourceDocCounter}; the runtime bean. {@code record} keeps two things in step: an
* hourly bucket ({@link SourceDocCountEntity}) that feeds the rolling 24h / 30d / daily-series
* windows, and a denormalized lifetime total ({@link SourceDocTotalEntity}) read directly for the
* all-time figure. The lifetime counter means the overview never scans a source's whole bucket
* history, and lets {@link #pruneOldBuckets()} retire buckets past the 30-day window so the hourly
* table stays bounded (~one row per source per active hour, for at most 30 days).
*/
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaSourceDocCounter implements SourceDocCounter {
private final SourceDocCountRepository countRepository;
private final SourceDocTotalRepository totalRepository;
private final Supplier<Instant> clock;
@Autowired
public JpaSourceDocCounter(
SourceDocCountRepository countRepository, SourceDocTotalRepository totalRepository) {
this(countRepository, totalRepository, Instant::now);
}
// Clock seam so tests can pin "now"; the runtime bean uses the wall clock above.
JpaSourceDocCounter(
SourceDocCountRepository countRepository,
SourceDocTotalRepository totalRepository,
Supplier<Instant> clock) {
this.countRepository = countRepository;
this.totalRepository = totalRepository;
this.clock = clock;
}
@Override
public void record(String sourceId, long docs) {
if (docs <= 0) {
return;
}
long bucketHour = currentHour();
upsert(
() -> totalRepository.increment(sourceId, docs),
() -> totalRepository.saveAndFlush(new SourceDocTotalEntity(sourceId, docs)));
upsert(
() -> countRepository.increment(sourceId, bucketHour, docs),
() ->
countRepository.saveAndFlush(
new SourceDocCountEntity(sourceId, bucketHour, docs)));
}
/**
* Add {@code docs} to a per-source running total: increment the existing row, else insert a new
* one. The insert is flushed now (in its own transaction, since {@code record} is not
* {@code @Transactional}) so a concurrent run's winning insert surfaces as a constraint
* violation we retry as an increment, rather than as a silent {@code merge} overwrite or a
* later doomed commit.
*/
private static void upsert(IntSupplier increment, Runnable insert) {
if (increment.getAsInt() > 0) {
return;
}
try {
insert.run();
} catch (DataIntegrityViolationException concurrentInsert) {
increment.getAsInt();
}
}
@Override
public Map<String, DocStats> statsFor(Collection<String> sourceIds) {
if (sourceIds.isEmpty()) {
return Map.of();
}
long now = currentHour();
Map<String, Long> totals = sums(totalRepository.totalsFor(sourceIds));
Map<String, Long> last24h =
sums(
countRepository.sumBySourceSince(
sourceIds, now - (SourceDocWindows.HOURS_IN_24H - 1)));
Map<String, Long> last30d =
sums(
countRepository.sumBySourceSince(
sourceIds, SourceDocWindows.firstDayHour(now)));
Map<String, DocStats> stats = new HashMap<>();
for (String id : sourceIds) {
stats.put(
id,
new DocStats(
totals.getOrDefault(id, 0L),
last24h.getOrDefault(id, 0L),
last30d.getOrDefault(id, 0L)));
}
return stats;
}
@Override
public List<Long> dailySeriesFor(String sourceId) {
long now = currentHour();
Collection<String> ids = List.of(sourceId);
Map<Long, Long> dailyCounts =
dailyBySource(
countRepository.dailyCountsSince(
ids, SourceDocWindows.firstDayHour(now)))
.getOrDefault(sourceId, Map.of());
return SourceDocWindows.series(dailyCounts, now / 24);
}
/**
* Retire hourly buckets older than the 30-day window; the lifetime total is held separately, so
* nothing reported is lost. Daily is ample - the read queries already ignore older buckets, so
* this is purely storage hygiene.
*/
@Scheduled(fixedDelay = 1, timeUnit = TimeUnit.DAYS)
public void pruneOldBuckets() {
countRepository.deleteOlderThan(SourceDocWindows.firstDayHour(currentHour()));
}
private long currentHour() {
return clock.get().getEpochSecond() / 3600;
}
private static Map<String, Long> sums(List<SourceDocSum> rows) {
Map<String, Long> map = new HashMap<>();
for (SourceDocSum row : rows) {
map.put(row.sourceId(), row.count() == null ? 0L : row.count());
}
return map;
}
private static Map<String, Map<Long, Long>> dailyBySource(List<SourceDayDocSum> rows) {
Map<String, Map<Long, Long>> bySource = new HashMap<>();
for (SourceDayDocSum row : rows) {
bySource.computeIfAbsent(row.sourceId(), key -> new HashMap<>())
.merge(row.day(), row.docs() == null ? 0L : row.docs(), Long::sum);
}
return bySource;
}
}
@@ -74,6 +74,20 @@ public class SourceController {
.orElseGet(() -> ResponseEntity.notFound().build());
}
@GetMapping("/{sourceId}/document-counts")
@Operation(
summary = "Daily document counts for a source",
description =
"The trailing 30-day per-day document series (oldest first) for the source's"
+ " sparkline.")
public ResponseEntity<List<Long>> documentCounts(@PathVariable String sourceId) {
return sourceStore
.get(sourceId)
.filter(sourceAccessGuard::canAccess)
.map(source -> ResponseEntity.ok(overviewService.dailySeries(source.id())))
.orElseGet(() -> ResponseEntity.notFound().build());
}
@PostMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Create or update a source",
@@ -0,0 +1,7 @@
package stirling.software.proprietary.policy.source;
/**
* A {@code (sourceId, epoch-day, summed document count)} row from the daily-aggregate query. The
* day is {@code floor(bucketHour / 24)}, i.e. hours-since-epoch collapsed to days-since-epoch.
*/
public record SourceDayDocSum(String sourceId, Long day, Long docs) {}
@@ -0,0 +1,62 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import org.springframework.data.domain.Persistable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* One hour's document tally for a source: {@code bucketHour} is the hours-since-epoch the documents
* were fed, {@code docCount} the running total for that hour. Rolling-window totals are summed from
* these buckets. {@code sourceId} is a plain value, not a foreign key, matching the rest of the
* subsystem so it stays decoupled from the security entities.
*/
@Entity
@Table(name = "policy_source_doc_counts")
@IdClass(SourceDocCountId.class)
@NoArgsConstructor
@Getter
@Setter
public class SourceDocCountEntity implements Serializable, Persistable<SourceDocCountId> {
private static final long serialVersionUID = 1L;
@Id
@Column(name = "source_id")
private String sourceId;
@Id
@Column(name = "bucket_hour")
private long bucketHour;
@Column(name = "doc_count")
private long docCount;
public SourceDocCountEntity(String sourceId, long bucketHour, long docCount) {
this.sourceId = sourceId;
this.bucketHour = bucketHour;
this.docCount = docCount;
}
@Override
@Transient
public SourceDocCountId getId() {
return new SourceDocCountId(sourceId, bucketHour);
}
@Override
@Transient
public boolean isNew() {
return true;
}
}
@@ -0,0 +1,36 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import java.util.Objects;
/** Composite key for {@link SourceDocCountEntity}: one row per source per hour bucket. */
public class SourceDocCountId implements Serializable {
private static final long serialVersionUID = 1L;
private String sourceId;
private long bucketHour;
public SourceDocCountId() {}
public SourceDocCountId(String sourceId, long bucketHour) {
this.sourceId = sourceId;
this.bucketHour = bucketHour;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SourceDocCountId other)) {
return false;
}
return bucketHour == other.bucketHour && Objects.equals(sourceId, other.sourceId);
}
@Override
public int hashCode() {
return Objects.hash(sourceId, bucketHour);
}
}
@@ -0,0 +1,69 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;
@Repository
public interface SourceDocCountRepository
extends JpaRepository<SourceDocCountEntity, SourceDocCountId> {
/**
* Add to an existing bucket; returns the number of rows updated (0 when the bucket is new).
* Transactional per call so {@code JpaSourceDocCounter.record} can run it (and the retry after
* a concurrent insert) without an enclosing transaction.
*/
@Modifying
@Transactional
@Query(
"update SourceDocCountEntity e set e.docCount = e.docCount + :docs"
+ " where e.sourceId = :sourceId and e.bucketHour = :bucketHour")
int increment(
@Param("sourceId") String sourceId,
@Param("bucketHour") long bucketHour,
@Param("docs") long docs);
/**
* Delete hourly buckets older than {@code floor} (hours-since-epoch). Nothing reads buckets
* before the 30-day window ({@code SourceDocWindows.firstDayHour}); the lifetime total lives in
* {@code policy_source_doc_totals}, so retiring old buckets keeps the table bounded without
* losing any reported figure.
*/
@Modifying
@Transactional
@Query("delete from SourceDocCountEntity e where e.bucketHour < :floor")
int deleteOlderThan(@Param("floor") long floor);
/**
* Document total per source restricted to buckets at or after {@code since} (the 24h window).
*/
@Query(
"select new stirling.software.proprietary.policy.source.SourceDocSum("
+ "e.sourceId, sum(e.docCount))"
+ " from SourceDocCountEntity e"
+ " where e.sourceId in :ids and e.bucketHour >= :since"
+ " group by e.sourceId")
List<SourceDocSum> sumBySourceSince(
@Param("ids") Collection<String> ids, @Param("since") long since);
/**
* Per-source, per-day document totals for buckets at or after {@code since}, summed in the
* database so the overview reads ~one row per source per active day instead of per active hour.
* The day is {@code cast(floor(bucketHour / 24.0) as long)}: {@code 24.0} forces decimal
* division and the cast pins the result to a whole day on every dialect.
*/
@Query(
"select new stirling.software.proprietary.policy.source.SourceDayDocSum("
+ "e.sourceId, cast(floor(e.bucketHour / 24.0) as long), sum(e.docCount))"
+ " from SourceDocCountEntity e"
+ " where e.sourceId in :ids and e.bucketHour >= :since"
+ " group by cast(floor(e.bucketHour / 24.0) as long), e.sourceId")
List<SourceDayDocSum> dailyCountsSince(
@Param("ids") Collection<String> ids, @Param("since") long since);
}
@@ -0,0 +1,28 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import java.util.Map;
/**
* Records and reports how many documents each source feeds into runs. Counting is bucketed by hour
* so the overview can report rolling totals ({@link DocStats}) cheaply; {@link JpaSourceDocCounter}
* is the runtime bean and {@link InProcessSourceDocCounter} backs tests.
*/
public interface SourceDocCounter {
/** Record that {@code docs} documents were fed from {@code sourceId} at the current time. */
void record(String sourceId, long docs);
/**
* Document totals for each given source; a source with no recorded docs maps to {@link
* DocStats#ZERO}.
*/
Map<String, DocStats> statsFor(Collection<String> sourceIds);
/**
* The trailing {@link DocStats#DAYS}-day daily document series for one source, oldest first,
* for the detail-panel sparkline.
*/
List<Long> dailySeriesFor(String sourceId);
}
@@ -0,0 +1,6 @@
package stirling.software.proprietary.policy.source;
/**
* A {@code (sourceId, summed document count)} row, populated by the doc-count aggregate queries.
*/
public record SourceDocSum(String sourceId, Long count) {}
@@ -0,0 +1,59 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import org.springframework.data.domain.Persistable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* A source's lifetime document total, denormalized so the overview reads the all-time count in one
* row instead of scanning the source's whole hourly-bucket history, and so {@link
* SourceDocCountEntity} buckets can be pruned to the rolling 30-day window without losing it.
*
* <p>Like {@link SourceDocCountEntity}, implements {@link Persistable} reporting {@code isNew() ==
* true} so a new source's first {@code save} {@code persist}s (a raw INSERT) and a concurrent
* insert surfaces as a constraint violation the counter retries as an increment, rather than {@code
* merge} silently overwriting it.
*/
@Entity
@Table(name = "policy_source_doc_totals")
@NoArgsConstructor
@Getter
@Setter
public class SourceDocTotalEntity implements Serializable, Persistable<String> {
private static final long serialVersionUID = 1L;
@Id
@Column(name = "source_id")
private String sourceId;
@Column(name = "doc_total")
private long docTotal;
public SourceDocTotalEntity(String sourceId, long docTotal) {
this.sourceId = sourceId;
this.docTotal = docTotal;
}
@Override
@Transient
public String getId() {
return sourceId;
}
@Override
@Transient
public boolean isNew() {
return true;
}
}
@@ -0,0 +1,34 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;
@Repository
public interface SourceDocTotalRepository extends JpaRepository<SourceDocTotalEntity, String> {
/**
* Add to a source's lifetime total; returns the number of rows updated (0 when the source has
* no total row yet). Transactional per call so {@code JpaSourceDocCounter.record} can run it
* (and the retry after a concurrent insert) without an enclosing transaction.
*/
@Modifying
@Transactional
@Query(
"update SourceDocTotalEntity e set e.docTotal = e.docTotal + :docs"
+ " where e.sourceId = :sourceId")
int increment(@Param("sourceId") String sourceId, @Param("docs") long docs);
/** Lifetime totals for the given sources, as {@code (sourceId, total)} rows. */
@Query(
"select new stirling.software.proprietary.policy.source.SourceDocSum("
+ "e.sourceId, e.docTotal)"
+ " from SourceDocTotalEntity e where e.sourceId in :ids")
List<SourceDocSum> totalsFor(@Param("ids") Collection<String> ids);
}
@@ -0,0 +1,61 @@
package stirling.software.proprietary.policy.source;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
* Shared rolling-window math over a source's document buckets, so the JPA and in-memory counters
* agree on the window boundaries and the daily series. Both define "last 30 days" as the buckets at
* or after {@link #firstDayHour}, and build the series from per-day counts with {@link #series}
* (the JPA counter aggregates by day in SQL; the in-memory counter groups its hourly buckets with
* {@link #byDay}).
*/
final class SourceDocWindows {
static final long HOURS_IN_24H = 24;
private SourceDocWindows() {}
/**
* The hours-since-epoch at the start of the oldest day in the 30-day window, given the current
* hour bucket. Both the "last 30 days" total and the daily series are measured from here, so
* the KPI and the sparkline always cover the same buckets.
*/
static long firstDayHour(long nowHour) {
return ((nowHour / 24) - (DocStats.DAYS - 1)) * 24;
}
/**
* Build the {@link DocStats#DAYS}-day daily series (oldest first) from per-day document counts
* (keyed by epoch-day, i.e. hours-since-epoch / 24). {@code currentDay} is today's epoch-day;
* the series runs back {@code DAYS} days from it.
*/
static List<Long> series(Map<Long, Long> dailyCounts, long currentDay) {
long firstDay = currentDay - (DocStats.DAYS - 1);
long[] daily = new long[DocStats.DAYS];
for (Map.Entry<Long, Long> day : dailyCounts.entrySet()) {
int dayIndex = (int) (day.getKey() - firstDay);
if (dayIndex >= 0 && dayIndex < DocStats.DAYS) {
daily[dayIndex] += day.getValue();
}
}
return Arrays.stream(daily).boxed().toList();
}
/** Collapse hourly buckets (keyed by hours-since-epoch) into per-day counts (keyed by day). */
static Map<Long, Long> byDay(Map<Long, Long> hourlyCounts) {
Map<Long, Long> dailyCounts = new HashMap<>();
hourlyCounts.forEach((hour, count) -> dailyCounts.merge(hour / 24, count, Long::sum));
return dailyCounts;
}
/** Sum hourly buckets at or after {@code since} (hours-since-epoch). */
static long sumSince(Map<Long, Long> hourlyCounts, long since) {
return hourlyCounts.entrySet().stream()
.filter(entry -> entry.getKey() >= since)
.mapToLong(Map.Entry::getValue)
.sum();
}
}
@@ -30,12 +30,15 @@ public class SourceOverviewService {
private final PolicyStore policyStore;
private final SourceAccessGuard sourceAccessGuard;
private final PolicyAccessGuard policyAccessGuard;
private final SourceDocCounter docCounter;
public SourcesResponse overview() {
List<Source> sources = sourceAccessGuard.visibleFrom(sourceStore);
List<Policy> policies = policyAccessGuard.visibleFrom(policyStore);
Map<String, List<Policy>> referencesBySource = referencesBySource(policies);
Map<String, DocStats> docStats =
docCounter.statsFor(sources.stream().map(Source::id).toList());
List<SourceView> views =
sources.stream()
@@ -44,7 +47,8 @@ public class SourceOverviewService {
toView(
source,
referencesBySource.getOrDefault(
source.id(), List.of())))
source.id(), List.of()),
docStats.getOrDefault(source.id(), DocStats.ZERO)))
.sorted(
Comparator.comparingInt(SourceView::referenceCount)
.reversed()
@@ -54,6 +58,14 @@ public class SourceOverviewService {
return new SourcesResponse(buildKpis(views), views);
}
/**
* The 30-day daily document series for one source (oldest first), for the expanded row's
* sparkline.
*/
public List<Long> dailySeries(String sourceId) {
return docCounter.dailySeriesFor(sourceId);
}
/** Policies referencing each source id, across the caller's visible policies. */
private static Map<String, List<Policy>> referencesBySource(List<Policy> policies) {
Map<String, List<Policy>> bySource = new HashMap<>();
@@ -65,7 +77,8 @@ public class SourceOverviewService {
return bySource;
}
private static SourceView toView(Source source, List<Policy> referencingPolicies) {
private static SourceView toView(
Source source, List<Policy> referencingPolicies, DocStats docs) {
List<SourceView.PolicyRef> refs =
referencingPolicies.stream()
.map(policy -> new SourceView.PolicyRef(policy.id(), policy.name()))
@@ -78,7 +91,9 @@ public class SourceOverviewService {
refs.size(),
refs,
configRows(source),
null);
docs.total(),
docs.last24h(),
docs.last30d());
}
/** A disabled (paused) source reads as "disabled"; an unreferenced one reads as "unused". */
@@ -4,8 +4,8 @@ import java.util.List;
/**
* One row in the Sources overview: a persisted input connection shown exactly once, with how many
* policies reference it (and which). {@code docsTotal} is {@code null} - per-source document volume
* is not tracked yet; the field is reserved so a later doc-accounting pass is additive.
* policies reference it (and which) and how many documents it has fed into runs ({@code docsTotal}
* lifetime plus the trailing 24-hour and 30-day windows).
*/
public record SourceView(
String id,
@@ -15,7 +15,9 @@ public record SourceView(
int referenceCount,
List<PolicyRef> referencingPolicies,
List<DetailRow> config,
Long docsTotal) {
long docsTotal,
long docs24h,
long docs30d) {
/** A policy that references this source. */
public record PolicyRef(String id, String name) {}
@@ -27,6 +27,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.policy.config.FolderAccessGuard;
import stirling.software.proprietary.policy.engine.PolicyRunner;
import stirling.software.proprietary.policy.input.InputSource;
import stirling.software.proprietary.policy.model.InputSpec;
@@ -74,6 +75,16 @@ public class FolderWatchTrigger implements PolicyTrigger {
return TYPE;
}
@Override
public boolean requiresSource() {
return true;
}
@Override
public Set<String> supportedSourceTypes() {
return Set.of(FolderAccessGuard.FOLDER_TYPE);
}
@Override
public void validate(Policy policy) {
if (watchDirsOf(policy).isEmpty()) {
@@ -1,5 +1,7 @@
package stirling.software.proprietary.policy.trigger;
import java.util.Set;
import stirling.software.proprietary.policy.model.Policy;
/**
@@ -11,6 +13,24 @@ public interface PolicyTrigger {
/** Matches {@code TriggerConfig.type()}. */
String type();
/**
* Whether this trigger needs at least one compatible input source to function. A schedule fires
* on the clock regardless of sources, so it is false; folder-watch derives the directories it
* watches from the policy's sources, so it is true. Drives whether the UI offers the trigger.
*/
default boolean requiresSource() {
return false;
}
/**
* The source {@code type()}s this trigger is compatible with (e.g. {@code "folder"}). Empty
* means source-agnostic (no constraint). Lets the UI offer a trigger only when a compatible
* source is selected, without hard-coding the relationship.
*/
default Set<String> supportedSourceTypes() {
return Set.of();
}
/**
* Validate at save time so misconfiguration fails fast, not at fire time. Receives the whole
* {@link Policy} so triggers that depend on the policy's sources (folder-watch) can check that.
@@ -0,0 +1,18 @@
package stirling.software.proprietary.policy.trigger;
import java.util.List;
/**
* Describes an available trigger for the admin UI: its {@code type} (matching {@code
* TriggerConfig.type()}), whether it needs a compatible source, and which source types it works
* with. Lets the UI list supported triggers and pair them with sources without hard-coding the set.
*/
public record TriggerInfo(String type, boolean requiresSource, List<String> supportedSourceTypes) {
public static TriggerInfo of(PolicyTrigger trigger) {
return new TriggerInfo(
trigger.type(),
trigger.requiresSource(),
List.copyOf(trigger.supportedSourceTypes()));
}
}
@@ -33,8 +33,11 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.storage.repository",
"stirling.software.proprietary.workflow.repository",
"stirling.software.proprietary.policy.store",
"stirling.software.proprietary.policy.source",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.policy.source"
"stirling.software.proprietary.access.repository",
"stirling.software.proprietary.integration.repository",
"stirling.software.proprietary.classification.store"
})
@EntityScan({
"stirling.software.proprietary.security.model",
@@ -42,8 +45,11 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.storage.model",
"stirling.software.proprietary.workflow.model",
"stirling.software.proprietary.policy.store",
"stirling.software.proprietary.policy.source",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.policy.source"
"stirling.software.proprietary.access.model",
"stirling.software.proprietary.integration.model",
"stirling.software.proprietary.classification.store"
})
public class DatabaseConfig {
@@ -28,6 +28,7 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.common.constants.JwtConstants;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.audit.AuditEventType;
import stirling.software.proprietary.audit.AuditLevel;
import stirling.software.proprietary.audit.Audited;
@@ -64,6 +65,7 @@ public class AuthController {
private final ApplicationProperties.Security securityProperties;
private final ApplicationProperties applicationProperties;
private final AiUserDataService aiUserDataService;
private final ResourceAccessService resourceAccessService;
/**
* Login endpoint - replaces Supabase signInWithPassword
@@ -628,6 +630,7 @@ public class AuthController {
userMap.put("username", user.getUsername());
userMap.put("role", user.getRolesAsString());
userMap.put("enabled", user.isEnabled());
userMap.put("portalAccess", resourceAccessService.canAccessPortal(user));
userMap.put(
"authenticationType",
user.getAuthenticationType()); // Expose authentication type for SSO detection
@@ -352,6 +352,7 @@ public class AiWorkflowService {
try {
List<Resource> resultFiles = new ArrayList<>();
List<Integer> origins = new ArrayList<>();
List<String> inputNames = new ArrayList<>();
for (int i = 0; i < filesToConvert.size(); i++) {
AiFile file = filesToConvert.get(i);
@@ -376,11 +377,15 @@ public class AiWorkflowService {
definition,
PolicyInputs.of(List.of(input)),
PolicyProgressListener.NOOP);
resultFiles.addAll(result.files());
// Each conversion runs on one input, so every output traces back to file i.
for (Resource output : result.files()) {
resultFiles.add(output);
origins.add(i);
}
inputNames.add(multipartFile.getOriginalFilename());
}
return new WorkflowState.Terminal(
buildCompletedResponse(null, resultFiles, inputNames, null));
buildCompletedResponse(null, resultFiles, origins, inputNames, null));
} catch (InternalApiTimeoutException e) {
log.error("PDF to Markdown conversion timed out: {}", e.getMessage());
return new WorkflowState.Terminal(
@@ -472,6 +477,7 @@ public class AiWorkflowService {
buildCompletedResponse(
response.getRationale(),
result.files(),
result.origins(),
inputFileNames(filesById),
result.report()));
} catch (InternalApiTimeoutException e) {
@@ -533,7 +539,8 @@ public class AiWorkflowService {
}
};
return new WorkflowState.Terminal(
buildCompletedResponse(response.getSummary(), List.of(resource), List.of(), null));
buildCompletedResponse(
response.getSummary(), List.of(resource), null, List.of(), null));
}
@SuppressWarnings("unchecked")
@@ -591,7 +598,11 @@ public class AiWorkflowService {
return new WorkflowState.Terminal(
buildCompletedResponse(
summary, result.files(), inputFileNames(filesById), result.report()));
summary,
result.files(),
result.origins(),
inputFileNames(filesById),
result.report()));
} catch (InternalApiTimeoutException e) {
log.error("Plan step on tool {} timed out: {}", e.getEndpointPath(), e.getMessage());
return new WorkflowState.Terminal(
@@ -680,19 +691,35 @@ public class AiWorkflowService {
private AiWorkflowResponse buildCompletedResponse(
String summary,
List<Resource> resultFiles,
List<Integer> origins,
List<String> inputFileNames,
JsonNode report)
throws IOException {
// Store every output file individually so each gets its own Stirling file ID and the
// frontend can add them as independent variants without going through a zip.
boolean preserveInputNames = inputFileNames.size() == resultFiles.size();
// Count outputs per source so only a clean 1:1 transform (one output for a source) reuses
// the input's name; a split (one input → many outputs) keeps each entry's own name.
Map<Integer, Long> outputsPerOrigin =
origins == null
? Map.of()
: origins.stream()
.filter(o -> o != null)
.collect(Collectors.groupingBy(o -> o, Collectors.counting()));
List<AiWorkflowResultFile> descriptors = new ArrayList<>();
for (int i = 0; i < resultFiles.size(); i++) {
Resource resource = resultFiles.get(i);
String responseName = resource.getFilename();
String inputName = preserveInputNames ? inputFileNames.get(i) : null;
// Prefer the input name only for 1:1 operations where the output keeps the same
// extension (rotate, compress, etc.). For converters and other extension-changing
// The output's source input (from the executor), used both to name it and to tell the
// client which file to version in place.
Integer origin = origins != null && i < origins.size() ? origins.get(i) : null;
boolean uniqueOrigin =
origin != null && outputsPerOrigin.getOrDefault(origin, 0L) == 1L;
String inputName =
uniqueOrigin && origin >= 0 && origin < inputFileNames.size()
? inputFileNames.get(origin)
: null;
// Prefer the source input's name only for 1:1 operations where the output keeps the
// same extension (rotate, compress, etc.). For converters and other extension-changing
// tools, the response filename from Content-Disposition is authoritative.
String name;
if (inputName != null
@@ -712,7 +739,11 @@ public class AiWorkflowService {
try (java.io.InputStream is = resource.getInputStream()) {
fileId = fileStorage.storeInputStream(is, name).fileId();
}
descriptors.add(new AiWorkflowResultFile(fileId, name, contentType));
// Only expose the source when this is a clean 1:1 transform, so the client can treat a
// present sourceIndex as "replace that input in place" without further disambiguation.
descriptors.add(
new AiWorkflowResultFile(
fileId, name, contentType, uniqueOrigin ? origin : null));
}
AiWorkflowResponse completed = new AiWorkflowResponse();
@@ -404,18 +404,16 @@ public class WorkflowParticipantController {
certSubmission.put("reason", request.getReason());
certSubmission.put("showLogo", request.getShowLogo());
// Store certificate files as base64
// Store the certificate keystores encrypted at rest.
if (request.getP12File() != null && !request.getP12File().isEmpty()) {
certSubmission.put(
"p12Keystore",
java.util.Base64.getEncoder()
.encodeToString(request.getP12File().getBytes()));
metadataEncryptionService.encryptBytes(request.getP12File().getBytes()));
}
if (request.getJksFile() != null && !request.getJksFile().isEmpty()) {
certSubmission.put(
"jksKeystore",
java.util.Base64.getEncoder()
.encodeToString(request.getJksFile().getBytes()));
metadataEncryptionService.encryptBytes(request.getJksFile().getBytes()));
}
metadata.put("certificateSubmission", certSubmission);
@@ -32,6 +32,7 @@ public class SignDocumentRequest {
private String password;
private MultipartFile privateKeyFile;
private MultipartFile certFile;
private MultipartFile jksFile;
// Signature metadata (participant can override owner defaults)
private String reason; // Participant's reason for signing
@@ -96,6 +96,28 @@ public class MetadataEncryptionService {
}
}
/**
* Encodes raw bytes to Base64 and encrypts them for at-rest storage (e.g. keystore files held
* in JSONB metadata). Returns {@code null} for {@code null} input.
*/
public String encryptBytes(byte[] data) {
if (data == null) {
return null;
}
return encrypt(Base64.getEncoder().encodeToString(data));
}
/**
* Reverses {@link #encryptBytes}. Also accepts legacy values stored as plain Base64 before
* encryption was introduced — {@link #decrypt} returns those unchanged, so they still decode.
*/
public byte[] decryptBytes(String stored) {
if (stored == null) {
return null;
}
return Base64.getDecoder().decode(decrypt(stored));
}
// ── Internals ───────────────────────────────────────────────────────────
private SecretKeySpec deriveKey() throws Exception {
@@ -1039,17 +1039,18 @@ public class SigningFinalizationService {
metadataEncryptionService.decrypt(submission.getPassword()));
}
// Decode base64 keystore bytes
// Decrypt + decode keystore bytes (supports both legacy plaintext base64 and
// encrypted values).
var certNode = node.get("certificateSubmission");
if (certNode.has("p12Keystore")) {
submission.setP12Keystore(
java.util.Base64.getDecoder()
.decode(certNode.get("p12Keystore").asText()));
metadataEncryptionService.decryptBytes(
certNode.get("p12Keystore").asText()));
}
if (certNode.has("jksKeystore")) {
submission.setJksKeystore(
java.util.Base64.getDecoder()
.decode(certNode.get("jksKeystore").asText()));
metadataEncryptionService.decryptBytes(
certNode.get("jksKeystore").asText()));
}
return submission;
}
@@ -1,6 +1,13 @@
package stirling.software.proprietary.workflow.service;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.HashMap;
@@ -8,6 +15,16 @@ import java.util.List;
import java.util.Map;
import java.util.UUID;
import org.bouncycastle.asn1.pkcs.PrivateKeyInfo;
import org.bouncycastle.openssl.PEMDecryptorProvider;
import org.bouncycastle.openssl.PEMEncryptedKeyPair;
import org.bouncycastle.openssl.PEMKeyPair;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;
import org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder;
import org.bouncycastle.openssl.jcajce.JcePEMDecryptorProviderBuilder;
import org.bouncycastle.operator.InputDecryptorProvider;
import org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@@ -705,24 +722,70 @@ public class WorkflowSessionService {
}
}
// 2. Store certificate submission data
Map<String, Object> certSubmission = new HashMap<>();
certSubmission.put("certType", request.getCertType());
certSubmission.put("password", metadataEncryptionService.encrypt(request.getPassword()));
// Store keystore files as base64 if provided
if (request.getP12File() != null && !request.getP12File().isEmpty()) {
// Validate an uploaded JKS keystore too (same early rejection as P12/PFX).
if ("JKS".equalsIgnoreCase(request.getCertType())
&& request.getJksFile() != null
&& !request.getJksFile().isEmpty()) {
try {
byte[] keystoreBytes = request.getP12File().getBytes();
String base64Keystore = java.util.Base64.getEncoder().encodeToString(keystoreBytes);
certSubmission.put("p12Keystore", base64Keystore);
certificateSubmissionValidator.validateAndExtractInfo(
request.getJksFile().getBytes(), "JKS", request.getPassword());
} catch (ResponseStatusException e) {
throw e;
} catch (IOException e) {
log.error("Failed to read P12 keystore file", e);
log.error("Failed to read JKS keystore file for validation", e);
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "Failed to process certificate file");
}
}
// 2. Store certificate submission data
Map<String, Object> certSubmission = new HashMap<>();
certSubmission.put("password", metadataEncryptionService.encrypt(request.getPassword()));
if ("PEM".equalsIgnoreCase(request.getCertType())) {
// PEM uploads are a separate private key + certificate, not a keystore. Convert them to
// a PKCS12 keystore here so finalization signs via the standard PKCS12 path.
byte[] p12 =
buildPkcs12FromPem(
request.getPrivateKeyFile(),
request.getCertFile(),
request.getPassword());
// Give PEM the same early validation (expiry, key recovery, test-sign) as uploaded
// PKCS12/JKS keystores, so an expired or unusable cert is rejected now rather than at
// finalization.
certificateSubmissionValidator.validateAndExtractInfo(
p12, "PKCS12", request.getPassword());
certSubmission.put("certType", "PKCS12");
// Store the keystore encrypted at rest.
certSubmission.put("p12Keystore", metadataEncryptionService.encryptBytes(p12));
} else {
certSubmission.put("certType", request.getCertType());
// Encrypt the uploaded keystore at rest: PKCS12/PFX → p12Keystore, JKS → jksKeystore.
if (request.getP12File() != null && !request.getP12File().isEmpty()) {
try {
certSubmission.put(
"p12Keystore",
metadataEncryptionService.encryptBytes(
request.getP12File().getBytes()));
} catch (IOException e) {
log.error("Failed to read P12 keystore file", e);
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "Failed to process certificate file");
}
} else if (request.getJksFile() != null && !request.getJksFile().isEmpty()) {
try {
certSubmission.put(
"jksKeystore",
metadataEncryptionService.encryptBytes(
request.getJksFile().getBytes()));
} catch (IOException e) {
log.error("Failed to read JKS keystore file", e);
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "Failed to process certificate file");
}
}
}
// Note: Signature appearance settings (showSignature, pageNumber, location, reason,
// showLogo)
// may already be in metadata if owner configured them when adding participant.
@@ -837,6 +900,69 @@ public class WorkflowSessionService {
"User is not a participant in this session"));
}
/**
* Converts an uploaded PEM private key + certificate into a PKCS12 keystore (protected with the
* supplied password) so finalization can sign via the standard PKCS12 path.
*/
private byte[] buildPkcs12FromPem(
MultipartFile privateKeyFile, MultipartFile certFile, String password) {
if (privateKeyFile == null
|| privateKeyFile.isEmpty()
|| certFile == null
|| certFile.isEmpty()) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,
"PEM signing requires both a private key file and a certificate file");
}
char[] pw = password != null ? password.toCharArray() : new char[0];
try {
PrivateKey privateKey = readPemPrivateKey(privateKeyFile.getBytes(), pw);
Certificate cert =
CertificateFactory.getInstance("X.509")
.generateCertificate(new ByteArrayInputStream(certFile.getBytes()));
KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(null, null);
keyStore.setKeyEntry("alias", privateKey, pw, new Certificate[] {cert});
ByteArrayOutputStream out = new ByteArrayOutputStream();
keyStore.store(out, pw);
return out.toByteArray();
} catch (ResponseStatusException e) {
throw e;
} catch (Exception e) {
log.error("Failed to build keystore from PEM certificate", e);
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,
"Failed to read PEM certificate — check the key/certificate files and password");
}
}
/** Reads a PEM private key (PKCS8/PKCS1, optionally password-encrypted). */
private PrivateKey readPemPrivateKey(byte[] pemBytes, char[] password) throws Exception {
try (PEMParser pemParser =
new PEMParser(new InputStreamReader(new ByteArrayInputStream(pemBytes)))) {
Object pemObject = pemParser.readObject();
JcaPEMKeyConverter converter = new JcaPEMKeyConverter().setProvider("BC");
PrivateKeyInfo keyInfo;
if (pemObject instanceof PKCS8EncryptedPrivateKeyInfo encrypted) {
InputDecryptorProvider decryptor =
new JceOpenSSLPKCS8DecryptorProviderBuilder().build(password);
keyInfo = encrypted.decryptPrivateKeyInfo(decryptor);
} else if (pemObject instanceof PEMEncryptedKeyPair encryptedKeyPair) {
PEMDecryptorProvider decryptor =
new JcePEMDecryptorProviderBuilder().build(password);
keyInfo = encryptedKeyPair.decryptKeyPair(decryptor).getPrivateKeyInfo();
} else if (pemObject instanceof PEMKeyPair keyPair) {
keyInfo = keyPair.getPrivateKeyInfo();
} else if (pemObject instanceof PrivateKeyInfo info) {
keyInfo = info;
} else {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "Unsupported PEM private key format");
}
return converter.getPrivateKey(keyInfo);
}
}
/** Helper class to wrap byte array as MultipartFile. */
private static class ByteArrayMultipartFile implements MultipartFile {
private final byte[] content;
@@ -0,0 +1,174 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import java.util.Optional;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.TeamRepository;
@ExtendWith(MockitoExtension.class)
class OwnershipServiceTest {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
@Mock private ResourceAccessService accessService;
@Mock private TeamLeadLookup teamLeadLookup;
@Mock private TeamRepository teamRepository;
@InjectMocks private OwnershipService ownership;
/** Minimal concrete OwnedResource for exercising the base behaviour. */
static class TestResource extends OwnedResource {
private final Long id;
TestResource(Long id) {
this.id = id;
}
@Override
public Long getId() {
return id;
}
}
// ---- scope-based create authorization ----
@Test
void userScopeSetsOwner() {
User user = user(7);
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.USER, null, user, () -> false);
assertThat(r.getScope()).isEqualTo(OwnerScope.USER);
assertThat(r.getOwnerUser()).isSameAs(user);
}
@Test
void userScopeBlockedByLockedServerOverride() {
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L), OwnerScope.USER, null, user(7), () -> true));
}
@Test
void adminMayCreateServerScope() {
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.SERVER, null, admin(1), () -> false);
assertThat(r.getOwnerUser()).isNull();
}
@Test
void nonAdminCannotCreateServerScope() {
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L),
OwnerScope.SERVER,
null,
user(7),
() -> false));
}
@Test
void teamLeaderMayCreateTeamScope() {
Team team = new Team();
team.setId(5L);
User user = user(7);
when(teamRepository.findById(5L)).thenReturn(Optional.of(team));
when(teamLeadLookup.isLeaderOfTeam(user, 5L)).thenReturn(true);
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.TEAM, 5L, user, () -> false);
assertThat(r.getOwnerTeam()).isSameAs(team);
}
@Test
void nonLeaderCannotCreateTeamScope() {
Team team = new Team();
team.setId(5L);
when(teamRepository.findById(5L)).thenReturn(Optional.of(team));
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L), OwnerScope.TEAM, 5L, user(7), () -> false));
}
// ---- use / manage ----
@Test
void enabledResourceUseDelegatesToTheAcl() {
TestResource r = new TestResource(1L);
User user = user(7);
when(accessService.canUseResource(any(), any(), any(), any(), any())).thenReturn(true);
assertThat(ownership.canUse(TYPE, r, user)).isTrue();
}
@Test
void disabledResourceUsableOnlyByOwnerOrAdmin() {
TestResource r = new TestResource(1L);
r.setEnabled(false);
r.setOwnerUser(user(7));
assertThat(ownership.canUse(TYPE, r, user(7))).isTrue(); // owner
assertThat(ownership.canUse(TYPE, r, user(8))).isFalse(); // someone else
assertThat(ownership.canUse(TYPE, r, admin(2))).isTrue(); // admin
}
@Test
void disabledTeamResourceUsableByLeaderOfOwningTeam() {
Team team = new Team();
team.setId(5L);
TestResource r = new TestResource(1L);
r.setEnabled(false);
r.setOwnerTeam(team);
User leader = user(7);
when(teamLeadLookup.isLeaderOfTeam(leader, 5L)).thenReturn(true);
assertThat(ownership.canUse(TYPE, r, leader)).isTrue(); // lead of the owning team
assertThat(ownership.canUse(TYPE, r, user(8))).isFalse(); // not a lead
}
// ---- helpers ----
private void assertForbidden(org.assertj.core.api.ThrowableAssert.ThrowingCallable call) {
assertThatThrownBy(call)
.isInstanceOf(ResponseStatusException.class)
.extracting(e -> ((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.FORBIDDEN);
}
private User user(long id) {
User u = new User();
u.setId(id);
u.setUsername("user" + id);
return u;
}
private User admin(long id) {
User u = user(id);
new Authority("ROLE_ADMIN", u);
return u;
}
}
@@ -0,0 +1,217 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.when;
import java.lang.reflect.Field;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.repository.ResourceGrantRepository;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
@ExtendWith(MockitoExtension.class)
class ResourceAccessServiceTest {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
private static final String RID = "42";
@Mock private ResourceGrantRepository grantRepository;
@Mock private TeamLeadLookup teamLeadLookup;
@InjectMocks private ResourceAccessService service;
@BeforeEach
void setPortalDefault() throws Exception {
Field f = ResourceAccessService.class.getDeclaredField("portalDefaultPolicy");
f.setAccessible(true);
f.set(service, DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS);
}
// ---- owner / admin short-circuits ----
@Test
void adminMayUseEvenWithExplicitOnlyAndNoGrants() {
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, admin(1)))
.isTrue();
}
@Test
void ownerMayUseEvenWithExplicitOnly() {
assertThat(
service.canUseResource(
TYPE, RID, 5L, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void nullUserIsAlwaysDenied() {
assertThat(service.canUseResource(TYPE, RID, 5L, DefaultAccessPolicy.ORG_ALL, null))
.isFalse();
assertThat(service.canManageResource(TYPE, RID, 5L, null)).isFalse();
}
// ---- explicit grants ----
@Test
void explicitUserGrantAllowsUse() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void teamGrantAllowsUseForTeamMember() {
stubGrants(grant(PrincipalType.TEAM, 7L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.EXPLICIT_ONLY,
userInTeam(5, 7)))
.isTrue();
}
@Test
void teamGrantDoesNotLeakToOtherTeams() {
stubGrants(grant(PrincipalType.TEAM, 7L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.EXPLICIT_ONLY,
userInTeam(5, 99)))
.isFalse();
}
@Test
void manageGrantImpliesUse() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.MANAGE));
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void useGrantDoesNotImplyManage() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.USE));
assertThat(service.canManageResource(TYPE, RID, null, user(5))).isFalse();
}
@Test
void manageGrantAllowsManage() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.MANAGE));
assertThat(service.canManageResource(TYPE, RID, null, user(5))).isTrue();
}
// ---- default policies ----
@Test
void orgAllDefaultAllowsAnyUser() {
stubGrants();
assertThat(service.canUseResource(TYPE, RID, null, DefaultAccessPolicy.ORG_ALL, user(5)))
.isTrue();
}
@Test
void explicitOnlyDefaultDeniesUngrantedUser() {
stubGrants();
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isFalse();
}
@Test
void teamLeadDefaultAllowsLeaderButNotRegularUser() {
stubGrants();
User leader = user(5);
when(teamLeadLookup.isAnyTeamLeader(leader)).thenReturn(true);
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS, leader))
.isTrue();
stubGrants();
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS,
user(6)))
.isFalse();
}
// ---- portal convenience (default policy ADMINS_AND_TEAM_LEADS) ----
@Test
void portalAccessibleByAdmin() {
assertThat(service.canAccessPortal(admin(1))).isTrue();
}
@Test
void portalDeniedToRegularUser() {
when(grantRepository.findByResourceTypeAndResourceId(ResourceType.PORTAL, ""))
.thenReturn(List.of());
assertThat(service.canAccessPortal(user(5))).isFalse();
}
// ---- helpers ----
private void stubGrants(ResourceGrant... grants) {
when(grantRepository.findByResourceTypeAndResourceId(TYPE, RID))
.thenReturn(List.of(grants));
}
private ResourceGrant grant(PrincipalType type, long principalId, AccessPermission permission) {
ResourceGrant g = new ResourceGrant();
g.setResourceType(TYPE);
g.setResourceId(RID);
g.setPrincipalType(type);
g.setPrincipalId(principalId);
g.setPermission(permission);
return g;
}
private User user(long id) {
User u = new User();
u.setId(id);
u.setUsername("user" + id);
return u;
}
private User admin(long id) {
User u = user(id);
new Authority("ROLE_ADMIN", u);
return u;
}
private User userInTeam(long id, long teamId) {
User u = user(id);
Team team = new Team();
team.setId(teamId);
u.setTeam(team);
return u;
}
}
@@ -0,0 +1,80 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import java.util.LinkedHashMap;
import java.util.Map;
import org.junit.jupiter.api.Test;
class SecretMaskerTest {
private final SecretMasker masker = new SecretMasker();
@Test
void maskHidesFlatAndNestedSecretsButKeepsPlainFields() {
Map<String, Object> nested = new LinkedHashMap<>();
nested.put("host", "h.example");
nested.put("secretKey", "sk-real");
Map<String, Object> config = new LinkedHashMap<>();
config.put("bucket", "acme");
config.put("accessKey", "AKIA123");
config.put("connection", nested); // non-sensitive parent, sensitive child
Map<String, Object> masked = masker.mask(config);
assertThat(masked.get("bucket")).isEqualTo("acme");
assertThat(masked.get("accessKey")).isEqualTo(SecretMasker.MASK);
@SuppressWarnings("unchecked")
Map<String, Object> conn = (Map<String, Object>) masked.get("connection");
assertThat(conn.get("host")).isEqualTo("h.example");
assertThat(conn.get("secretKey")).isEqualTo(SecretMasker.MASK);
}
@Test
void mergeKeepsStoredSecretWhenIncomingIsMasked() {
Map<String, Object> stored = Map.of("bucket", "old", "secretKey", "REAL");
Map<String, Object> incoming = Map.of("bucket", "new", "secretKey", SecretMasker.MASK);
Map<String, Object> merged = masker.merge(stored, incoming);
assertThat(merged.get("bucket")).isEqualTo("new"); // non-secret updated
assertThat(merged.get("secretKey")).isEqualTo("REAL"); // secret retained
}
@Test
void mergeAcceptsARealNewSecret() {
Map<String, Object> merged =
masker.merge(Map.of("secretKey", "OLD"), Map.of("secretKey", "NEW"));
assertThat(merged.get("secretKey")).isEqualTo("NEW");
}
@Test
void sanitizeDropsBlankSecretsOnCreate() {
Map<String, Object> incoming = new LinkedHashMap<>();
incoming.put("bucket", "b");
incoming.put("secretKey", "");
Map<String, Object> clean = masker.sanitize(incoming);
assertThat(clean).containsEntry("bucket", "b").doesNotContainKey("secretKey");
}
@Test
void deeplyNestedInputIsBoundedNotOverflowing() {
// Build a structure far deeper than the recursion cap.
Map<String, Object> root = new LinkedHashMap<>();
Map<String, Object> cur = root;
for (int i = 0; i < 2000; i++) {
Map<String, Object> next = new LinkedHashMap<>();
cur.put("child", next);
cur = next;
}
cur.put("secretKey", "deep");
// Must return (bounded recursion), not throw StackOverflowError.
assertThat(masker.mask(root)).isNotNull();
assertThat(masker.sanitize(root)).isNotNull();
assertThat(masker.merge(root, root)).isNotNull();
}
}

Some files were not shown because too many files have changed in this diff Show More