Compare commits

..
Author SHA1 Message Date
Anthony Stirling 38ccea074c Version bump 2026-07-08 10:50:37 +01:00
Anthony Stirling a7307ff393 Fix Postgres user settings for some users 2026-07-08 10:50:36 +01:00
Anthony Stirling 72729e99c1 fix(release): stop msiexec hang in Windows signature verify; don't force latest or regen release notes 2026-07-07 23:35:12 +01:00
Anthony Stirling 5fba2720f0 Fix cert sign not showing under certain instances (#6908) 2026-07-07 22:45:56 +01:00
01a1ef8c44 Fix missing app icon on Linux/Wayland (#6875)
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-07-07 22:15:01 +01:00
LudyandAnthony Stirling 8535c7e9ac feat(ui): add dedicated third-party license sections to settings (#6820) 2026-07-07 22:15:01 +01:00
ConnorYohandAnthony Stirling cca3f42623 Set App version to v2.14.1 (#6891)
Upped version in build.gradle then ran build so version falls through
2026-07-07 12:15:01 +01:00
James BruntonandAnthony Stirling 1b7ffcdbac Fix tooltip positioning on Add Page Numbers (#6885)
# Description of Changes
## Before

<img width="483" height="227" alt="image"
src="https://github.com/user-attachments/assets/4bf86eec-a9cc-4f63-84f0-4eb2bd535bab"
/>

## After

<img width="732" height="235" alt="image"
src="https://github.com/user-attachments/assets/101d2ea4-36e8-4e8f-990a-d72b33fa0ac2"
/>
2026-07-07 12:09:51 +01:00
LudyandAnthony Stirling 67a0ca6110 fix(frontend): respect analytics config before initializing PostHog (#6812)
# Description of Changes

Please provide a summary of the changes, including:

- What was changed
- Moved PostHog startup out of `index.tsx` and into a config-aware
initializer inside `AppProviders`.
- Added a dedicated `usePosthogTracking` hook that only initializes
PostHog when `enableAnalytics` is explicitly `true` and `enablePosthog`
is not disabled.
- Kept cookie-consent handling in the same flow so consent is applied
only after PostHog is actually initialized.
- Removed the unconditional `PostHogProvider` and `posthog.init(...)`
bootstrap from the app entrypoint.
- Added targeted frontend tests covering analytics-disabled and
analytics-enabled startup behavior.

- Why the change was made
- The previous frontend bootstrap initialized PostHog before app config
was loaded, so disabling analytics in the UI or via environment settings
did not prevent PostHog network activity.
- This change makes analytics behavior follow the server-provided config
instead of always connecting on page load.

Closes #6358

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-07 12:09:51 +01:00
Anthony Stirling 8e4b2e2fc6 Disable update check and notification in SaaS mode (#6863)
# Description of Changes

In SaaS mode the self-hosted "Update Available" notification could still
appear and the update-check code (external call to
`supabase.stirling.com/functions/v1/updates`) still ran, even though the
cloud owns app versioning. The web `UpdateStartupPopup` was already
SaaS-gated via a null override, but two other paths were not:

- **Desktop app in SaaS connection mode** - `useDesktopUpdatePopup()`
ran its startup check and rendered the `UpdateModal` regardless of
connection mode, so a self-hosted update popup appeared while connected
to SaaS.
- **Settings → General** - the core `GeneralSection` fired
`checkForUpdate()` on mount unconditionally, even when the update
section was hidden (as SaaS does), so the external call still ran.

**What changed**

- `useDesktopUpdatePopup.ts` - the startup timer now bails out
immediately when `connectionModeService.getCurrentMode() === "saas"`. No
mode lookup, no external fetch, no modal.
- `core/GeneralSection.tsx` - the mount `checkForUpdate()` now returns
early when `hideUpdateSection` is set, so hiding the section (web SaaS,
managed-disabled desktop) also stops the external call.
- `desktop/GeneralSection.tsx` - passes `hideUpdateSection` when
`useSaaSMode()` is true, which (via the above) suppresses the settings
check in desktop-SaaS too.

**Why** - in SaaS the update check should never be called and no update
notification should be shown; the cloud handles versioning.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-07 12:09:51 +01:00
Anthony Stirling 3c93457021 Fix rearrange-pages DUPLICATE producing shared page nodes (pypdf cyclic-references CI break) (#6851) 2026-07-07 12:09:51 +01:00
230 changed files with 2134 additions and 13064 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.14.0
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.14.0
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
+23 -6
View File
@@ -510,6 +510,7 @@ jobs:
# cargo output unsigned, so checking it produces false negatives.
- name: Verify Windows Code Signature
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
timeout-minutes: 15
shell: pwsh
run: |
$allSigned = $true
@@ -531,11 +532,26 @@ jobs:
# Extract MSI and verify the inner exe (the file that actually gets installed).
# This is the critical check - AV flags the installed exe at runtime.
# Use lessmsi, not `msiexec /a`: msiexec serializes on the global
# _MSIExecute mutex and hangs forever on hosted runners when another
# installer is busy. lessmsi reads MSI tables directly - no mutex, no service.
$msi = $msiFiles[0].FullName
$extractDir = Join-Path $env:RUNNER_TEMP "msi-verify"
if (Test-Path $extractDir) { Remove-Item $extractDir -Recurse -Force }
$proc = Start-Process msiexec.exe -ArgumentList '/a', $msi, '/qn', "TARGETDIR=$extractDir" -Wait -PassThru -NoNewWindow
if ($proc.ExitCode -eq 0) {
New-Item -ItemType Directory -Force -Path $extractDir | Out-Null
choco install lessmsi -y --no-progress --limit-output | Out-Null
# Bound the extraction and kill on hang (defence in depth over timeout-minutes).
$proc = Start-Process lessmsi -ArgumentList 'x', "`"$msi`"", "`"$extractDir\`"" -PassThru -NoNewWindow
if (-not $proc.WaitForExit(120000)) {
try { $proc.Kill() } catch {}
Write-Host "[ERROR] MSI extraction timed out after 120s"
$allSigned = $false
} elseif ($proc.ExitCode -ne 0) {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
} else {
$innerExe = Get-ChildItem -Path $extractDir -Filter "stirling-pdf.exe" -Recurse -File | Select-Object -First 1
if ($innerExe) {
$sig = Get-AuthenticodeSignature -FilePath $innerExe.FullName
@@ -548,9 +564,6 @@ jobs:
Write-Host "[ERROR] Could not find stirling-pdf.exe inside MSI"
$allSigned = $false
}
} else {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
}
if (-not $allSigned) {
@@ -800,7 +813,11 @@ jobs:
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
# Don't regenerate/append notes on re-runs, and don't force this into the
# "Latest" slot - leave the release body and latest marker as they are.
generate_release_notes: false
append_body: false
make_latest: false
fail_on_unmatched_files: true
# Installers + updater payloads + manifest. .sig contents are embedded
# in latest.json so the .sig files themselves are not uploaded.
-21
View File
@@ -55,15 +55,6 @@ tasks:
- editor/src/core/data/ogImageMap.json
- editor/public/og-metadata.json
prepare:classifier-categories:
internal: true
run: when_changed
desc: "Regenerate the engine classifier categories JSON from the TS source of truth"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts
sources:
- editor/src/proprietary/data/classificationTaxonomy.ts
prepare:
desc: "Set up dev environment"
run: when_changed
@@ -74,7 +65,6 @@ tasks:
vars: { MODE: '{{.MODE}}' }
- prepare:icons
- prepare:og
- prepare:classifier-categories
# ============================================================
# Development
@@ -411,23 +401,12 @@ tasks:
cmds:
- node editor/scripts/generate-og-metadata.mjs --check
classifier-categories:
desc: "Regenerate the engine classifier categories JSON from the TS source"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts
classifier-categories:check:
desc: "Fail if the committed classifier categories JSON is out of date"
cmds:
- npx tsx editor/scripts/generate-classification-taxonomy.mts --check
check:all:
desc: "Full CI quality gate"
cmds:
# Runs first, before prepare regenerates: guards the committed og-metadata.json /
# ogImageMap.json that the Cloudflare Pages (plain `vite build`) deploy relies on.
- task: og:check
- task: classifier-categories:check
- task: typecheck:all
- task: lint
- task: format:check
-27
View File
@@ -95,7 +95,6 @@ tasks:
vars:
PORT: '{{.PORTAL_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
MOCKS: 'false'
OPEN: "true"
dev:portal:all:
@@ -118,32 +117,6 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
# Point the portal's "Editor" app switcher at the editor we spawn here.
EDITOR_URL: 'http://localhost:{{.EDITOR_PORT}}/'
MOCKS: 'false'
OPEN: "true"
- task: frontend:dev
vars:
PORT: '{{.EDITOR_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
dev:portal:all:saas:
desc: "Start SaaS backend + developer portal + editor concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5174{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5174{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
PORTAL_PORT: '{{index (splitList "\n" .PORTS) 1}}'
EDITOR_PORT: '{{index (splitList "\n" .PORTS) 2}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
POLICIES_ENABLED: "true"
- task: frontend:dev:portal
vars:
PORT: '{{.PORTAL_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
EDITOR_URL: 'http://localhost:{{.EDITOR_PORT}}/'
MOCKS: 'false'
OPEN: "true"
- task: frontend:dev
vars:
@@ -7,7 +7,6 @@ import java.time.format.DateTimeFormatter;
import java.util.Calendar;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
@@ -18,11 +17,6 @@ import stirling.software.common.model.PdfMetadata;
@Service
public class PdfMetadataService {
/**
* ({@code {category, docType, typeConfidence, tags}}). Written by the classify-and-tag tool.
*/
public static final String CLASSIFICATION_KEY = "StirlingPDFClassification";
private final ApplicationProperties applicationProperties;
private final String stirlingPDFLabel;
private final UserServiceInterface userService;
@@ -183,14 +177,4 @@ public class PdfMetadataService {
}
pdf.getDocumentInformation().setAuthor(author);
}
/**
* Write the document classifier's JSON result into the custom Info-dictionary field {@link
* #CLASSIFICATION_KEY}, leaving all other metadata untouched.
*/
public void setClassificationMetadata(PDDocument pdf, String classificationJson) {
PDDocumentInformation info = pdf.getDocumentInformation();
info.setCustomMetadataValue(CLASSIFICATION_KEY, classificationJson);
pdf.setDocumentInformation(info);
}
}
@@ -305,23 +305,6 @@ public class GetInfoOnPDF {
}
}
/**
* Info-dictionary keys exposed above via typed getters; any other key in the dictionary is
* surfaced as custom metadata (e.g. the classification policy's StirlingPDFClassification
* entry).
*/
private static final java.util.Set<String> STANDARD_INFO_KEYS =
java.util.Set.of(
"Title",
"Author",
"Subject",
"Keywords",
"Producer",
"Creator",
"CreationDate",
"ModDate",
"Trapped");
private static ObjectNode extractMetadata(PDDocument document) {
ObjectNode metadata = objectMapper.createObjectNode();
@@ -352,18 +335,6 @@ public class GetInfoOnPDF {
if (modificationDate != null) {
metadata.put("ModificationDate", modificationDate);
}
// Surface custom Info-dictionary entries (anything beyond the
// standard fields above) — e.g. StirlingPDFClassification
for (String key : info.getMetadataKeys()) {
if (STANDARD_INFO_KEYS.contains(key)) {
continue;
}
String value = info.getCustomMetadataValue(key);
if (value != null && !value.isBlank()) {
metadata.put(key, value);
}
}
}
} catch (Exception e) {
log.error("Error extracting metadata: {}", e.getMessage());
-4
View File
@@ -80,10 +80,6 @@ dependencies {
implementation "software.amazon.awssdk:s3:${awsSdkVersion}"
implementation "software.amazon.awssdk:url-connection-client:${awsSdkVersion}"
// @DataJpaTest slice (Boot 4 ships test slices as separate starters, like webmvc-test at the
// root) so policy.source repositories can be exercised against embedded H2.
testImplementation 'org.springframework.boot:spring-boot-starter-data-jpa-test'
// Testcontainers: real MinIO/LocalStack (S3) and Valkey for integration tests in CI without
// manually-started instances. Tests skip cleanly when Docker is unavailable.
testImplementation "org.testcontainers:testcontainers:${testcontainersMinioVersion}"
@@ -1,20 +0,0 @@
package stirling.software.proprietary.access.config;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.proprietary.access.service.DefaultTeamLeadLookup;
import stirling.software.proprietary.access.service.TeamLeadLookup;
/** Access-layer bean wiring. */
@Configuration
public class AccessConfig {
/** No-op {@link TeamLeadLookup} unless another bean is defined. */
@Bean
@ConditionalOnMissingBean(TeamLeadLookup.class)
TeamLeadLookup defaultTeamLeadLookup() {
return new DefaultTeamLeadLookup();
}
}
@@ -1,101 +0,0 @@
package stirling.software.proprietary.access.controller;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.User;
/** Admin endpoints to grant/revoke access to gated resources (the portal, integration configs). */
@RestController
@RequestMapping("/api/v1/admin/access")
@RequiredArgsConstructor
@PreAuthorize("hasRole('ADMIN')")
@Tag(name = "Access Control", description = "Manage resource access grants (portal, integrations)")
public class ResourceGrantController {
private final ResourceAccessService accessService;
@GetMapping("/grants")
public ResponseEntity<?> list(
@RequestParam ResourceType resourceType,
@RequestParam(required = false, defaultValue = "") String resourceId) {
List<ResourceGrant> grants = accessService.listGrants(resourceType, resourceId);
return ResponseEntity.ok(grants.stream().map(this::toDto).toList());
}
@PostMapping("/grants")
public ResponseEntity<?> create(
@RequestBody GrantRequest request, @AuthenticationPrincipal User admin) {
if (request.resourceType() == null
|| request.principalType() == null
|| request.principalId() == null) {
return ResponseEntity.badRequest()
.body(
Map.of(
"error",
"resourceType, principalType and principalId are required"));
}
AccessPermission permission =
request.permission() == null ? AccessPermission.USE : request.permission();
// PORTAL is a singleton resource; its grants always target the whole type.
String resourceId =
request.resourceType() == ResourceType.PORTAL ? "" : request.resourceId();
ResourceGrant grant =
accessService.grant(
request.resourceType(),
resourceId,
request.principalType(),
request.principalId(),
permission,
admin);
return ResponseEntity.ok(toDto(grant));
}
@DeleteMapping("/grants/{id}")
public ResponseEntity<?> delete(@PathVariable Long id) {
accessService.revoke(id);
return ResponseEntity.ok(Map.of("message", "Grant revoked"));
}
private Map<String, Object> toDto(ResourceGrant g) {
Map<String, Object> m = new HashMap<>();
m.put("id", g.getId());
m.put("resourceType", g.getResourceType());
m.put("resourceId", g.getResourceId());
m.put("principalType", g.getPrincipalType());
m.put("principalId", g.getPrincipalId());
m.put("permission", g.getPermission());
m.put("createdAt", g.getCreatedAt());
return m;
}
/** Request body for creating a grant. */
public record GrantRequest(
ResourceType resourceType,
String resourceId,
PrincipalType principalType,
Long principalId,
AccessPermission permission) {}
}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.access.model;
/** Permission level a grant confers. MANAGE implies USE. */
public enum AccessPermission {
USE,
MANAGE
}
@@ -1,14 +0,0 @@
package stirling.software.proprietary.access.model;
/**
* Fallback policy applied when no explicit {@link ResourceGrant} matches. Admins (org owners)
* always pass regardless of this policy.
*/
public enum DefaultAccessPolicy {
// Every authenticated user in the deployment (org) may use the resource.
ORG_ALL,
// Only org admins and team leaders. This is the default for the portal.
ADMINS_AND_TEAM_LEADS,
// Nobody but the owner, admins, and explicit grantees.
EXPLICIT_ONLY
}
@@ -1,57 +0,0 @@
package stirling.software.proprietary.access.model;
import jakarta.persistence.Column;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.FetchType;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.MappedSuperclass;
import lombok.Getter;
import lombok.Setter;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
/** Base for a resource owned by a user, a team, or the server, with grant-based access. */
@MappedSuperclass
@Getter
@Setter
public abstract class OwnedResource {
@Enumerated(EnumType.STRING)
@Column(name = "scope", nullable = false, length = 32)
private OwnerScope scope;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "owner_user_id")
private User ownerUser;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "owner_team_id")
private Team ownerTeam;
@Column(name = "enabled", nullable = false)
private boolean enabled = true;
// Server resource that users cannot override with their own of the same kind.
@Column(name = "locked", nullable = false)
private boolean locked = false;
// Who, besides owner/admin/grantees, may use this resource.
@Enumerated(EnumType.STRING)
@Column(name = "default_access", nullable = false, length = 32)
private DefaultAccessPolicy defaultAccess = DefaultAccessPolicy.EXPLICIT_ONLY;
/** Subclass primary key. */
public abstract Long getId();
public Long getOwnerUserId() {
return ownerUser != null ? ownerUser.getId() : null;
}
public Long getOwnerTeamId() {
return ownerTeam != null ? ownerTeam.getId() : null;
}
}
@@ -1,8 +0,0 @@
package stirling.software.proprietary.access.model;
/** Ownership scope of an {@link OwnedResource}: a single user, a team, or the whole server. */
public enum OwnerScope {
USER,
TEAM,
SERVER
}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.access.model;
/** Who a {@link ResourceGrant} is granted to. Org-wide access is expressed via default policy. */
public enum PrincipalType {
USER,
TEAM
}
@@ -1,86 +0,0 @@
package stirling.software.proprietary.access.model;
import java.io.Serializable;
import java.time.LocalDateTime;
import org.hibernate.annotations.CreationTimestamp;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Index;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.persistence.UniqueConstraint;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.security.model.User;
/** Grants a user or team access to a resource. Owner and admin access are implicit. */
@Entity
@Table(
name = "resource_grants",
uniqueConstraints =
@UniqueConstraint(
name = "uk_resource_grant",
columnNames = {
"resource_type",
"resource_id",
"principal_type",
"principal_id",
"permission"
}),
indexes = {
@Index(name = "idx_resource_grants_lookup", columnList = "resource_type,resource_id"),
@Index(
name = "idx_resource_grants_principal",
columnList = "principal_type,principal_id")
})
@NoArgsConstructor
@Getter
@Setter
public class ResourceGrant implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "resource_grant_id")
private Long id;
@Enumerated(EnumType.STRING)
@Column(name = "resource_type", nullable = false, length = 64)
private ResourceType resourceType;
// Empty string (never null) for a whole-type grant such as the portal.
@Column(name = "resource_id", nullable = false, length = 255)
private String resourceId = "";
@Enumerated(EnumType.STRING)
@Column(name = "principal_type", nullable = false, length = 32)
private PrincipalType principalType;
@Column(name = "principal_id", nullable = false)
private Long principalId;
@Enumerated(EnumType.STRING)
@Column(name = "permission", nullable = false, length = 32)
private AccessPermission permission;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "granted_by_user_id")
private User grantedBy;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
}
@@ -1,9 +0,0 @@
package stirling.software.proprietary.access.model;
/** Types of resources whose access can be gated by {@link ResourceGrant}. */
public enum ResourceType {
// The admin portal / processor (frontend/portal). Singleton resource (empty resourceId).
PORTAL,
// A stored S3/MCP/API integration configuration.
INTEGRATION_CONFIG
}
@@ -1,28 +0,0 @@
package stirling.software.proprietary.access.repository;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
@Repository
public interface ResourceGrantRepository extends JpaRepository<ResourceGrant, Long> {
List<ResourceGrant> findByResourceTypeAndResourceId(
ResourceType resourceType, String resourceId);
List<ResourceGrant> findByResourceTypeAndPrincipalTypeAndPrincipalId(
ResourceType resourceType, PrincipalType principalType, Long principalId);
void deleteByResourceTypeAndResourceId(ResourceType resourceType, String resourceId);
boolean existsByResourceTypeAndResourceIdAndPrincipalTypeAndPrincipalId(
ResourceType resourceType,
String resourceId,
PrincipalType principalType,
Long principalId);
}
@@ -1,44 +0,0 @@
package stirling.software.proprietary.access.security;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
/** {@code @PreAuthorize} bean for portal-access checks. Active in self-hosted and saas. */
@Component("resourceAccess")
@RequiredArgsConstructor
public class ResourceAccessSecurity {
private final ResourceAccessService accessService;
private final UserService userService;
public boolean canUsePortal() {
User user = currentUser();
return user != null && accessService.canAccessPortal(user);
}
private User currentUser() {
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
if (auth == null || !auth.isAuthenticated()) {
return null;
}
Object principal = auth.getPrincipal();
if (principal instanceof User user) {
return user;
}
if (principal instanceof UserDetails userDetails) {
return userService.findByUsername(userDetails.getUsername()).orElse(null);
}
if (principal instanceof String username && !"anonymousUser".equals(username)) {
return userService.findByUsername(username).orElse(null);
}
return null;
}
}
@@ -1,17 +0,0 @@
package stirling.software.proprietary.access.service;
import stirling.software.proprietary.security.model.User;
/** No-op {@link TeamLeadLookup}: always false. */
public class DefaultTeamLeadLookup implements TeamLeadLookup {
@Override
public boolean isAnyTeamLeader(User user) {
return false;
}
@Override
public boolean isLeaderOfTeam(User user, Long teamId) {
return false;
}
}
@@ -1,116 +0,0 @@
package stirling.software.proprietary.access.service;
import java.util.Set;
import java.util.function.BooleanSupplier;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.server.ResponseStatusException;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.TeamRepository;
/** Ownership and access checks for {@link OwnedResource}, backed by the resource-grant ACL. */
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class OwnershipService {
private final ResourceAccessService accessService;
private final TeamLeadLookup teamLeadLookup;
private final TeamRepository teamRepository;
/** Whether the user may use the resource. */
public boolean canUse(ResourceType type, OwnedResource resource, User user) {
if (!resource.isEnabled()) {
return isAdmin(user) || isOwner(resource, user);
}
return accessService.canUseResource(
type,
String.valueOf(resource.getId()),
resource.getOwnerUserId(),
resource.getDefaultAccess(),
user);
}
/** Whether the user may manage the resource. */
public boolean canManage(ResourceType type, OwnedResource resource, User user) {
return accessService.canManageResource(
type, String.valueOf(resource.getId()), resource.getOwnerUserId(), user);
}
/**
* Authorizes the scope and assigns ownership; {@code lockedOverrideBlocks} guards USER scope.
*/
public void assignOwnership(
OwnedResource resource,
OwnerScope scope,
Long teamId,
User user,
BooleanSupplier lockedOverrideBlocks) {
resource.setScope(scope);
switch (scope) {
case USER -> {
if (lockedOverrideBlocks.getAsBoolean() && !isAdmin(user)) {
throw forbidden(
"This is locked to the server configuration by an administrator");
}
resource.setOwnerUser(user);
}
case SERVER -> {
if (!isAdmin(user)) {
throw forbidden("Only administrators can create server-owned resources");
}
}
case TEAM -> {
if (teamId == null) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST, "ownerTeamId is required");
}
Team team =
teamRepository
.findById(teamId)
.orElseThrow(() -> notFound("Team not found"));
if (!isAdmin(user) && !teamLeadLookup.isLeaderOfTeam(user, team.getId())) {
throw forbidden("Only admins or team leaders can create team-owned resources");
}
resource.setOwnerTeam(team);
}
}
}
/** Resource ids of the given type the user or their team holds a grant on. */
public Set<String> grantedResourceIds(ResourceType type, User user) {
return accessService.grantedResourceIds(type, user);
}
public boolean isAdmin(User user) {
return user.getAuthorities().stream()
.anyMatch(a -> Role.ADMIN.getRoleId().equals(a.getAuthority()));
}
public boolean isOwner(OwnedResource resource, User user) {
if (resource.getOwnerUserId() != null && resource.getOwnerUserId().equals(user.getId())) {
return true;
}
// Team-owned: the lead of the owning team owns it.
return resource.getOwnerTeamId() != null
&& teamLeadLookup.isLeaderOfTeam(user, resource.getOwnerTeamId());
}
private ResponseStatusException forbidden(String message) {
return new ResponseStatusException(HttpStatus.FORBIDDEN, message);
}
private ResponseStatusException notFound(String message) {
return new ResponseStatusException(HttpStatus.NOT_FOUND, message);
}
}
@@ -1,188 +0,0 @@
package stirling.software.proprietary.access.service;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.repository.ResourceGrantRepository;
import stirling.software.proprietary.security.model.User;
/** Resolves access to gated resources: owner, then admin, then grant, then default policy. */
@Service
@RequiredArgsConstructor
@Slf4j
@Transactional(readOnly = true)
public class ResourceAccessService {
private final ResourceGrantRepository grantRepository;
private final TeamLeadLookup teamLeadLookup;
@Value("${security.portal.defaultAccess:ADMINS_AND_TEAM_LEADS}")
private DefaultAccessPolicy portalDefaultPolicy;
// ---- public checks ----
/** Whether the user may use the portal / processor. */
public boolean canAccessPortal(User user) {
return canUseResource(ResourceType.PORTAL, "", null, portalDefaultPolicy, user);
}
/** Whether the user may use a resource, falling back to its default policy. */
public boolean canUseResource(
ResourceType type,
String resourceId,
Long ownerUserId,
DefaultAccessPolicy defaultPolicy,
User user) {
if (user == null) {
return false;
}
if (isOwner(ownerUserId, user) || isAdmin(user)) {
return true;
}
if (hasGrant(type, normalize(resourceId), user, AccessPermission.USE)) {
return true;
}
return matchesDefault(defaultPolicy, user);
}
/** Whether the user may manage (edit/delete/share) a resource. No default-policy fallback. */
public boolean canManageResource(
ResourceType type, String resourceId, Long ownerUserId, User user) {
if (user == null) {
return false;
}
if (isOwner(ownerUserId, user) || isAdmin(user)) {
return true;
}
return hasGrant(type, normalize(resourceId), user, AccessPermission.MANAGE);
}
// ---- grant management ----
@Transactional
public ResourceGrant grant(
ResourceType type,
String resourceId,
PrincipalType principalType,
Long principalId,
AccessPermission permission,
User grantedBy) {
String rid = normalize(resourceId);
ResourceGrant grant =
grantRepository.findByResourceTypeAndResourceId(type, rid).stream()
.filter(
g ->
g.getPrincipalType() == principalType
&& g.getPrincipalId().equals(principalId))
.findFirst()
.orElseGet(ResourceGrant::new);
grant.setResourceType(type);
grant.setResourceId(rid);
grant.setPrincipalType(principalType);
grant.setPrincipalId(principalId);
grant.setPermission(permission);
if (grantedBy != null) {
grant.setGrantedBy(grantedBy);
}
return grantRepository.save(grant);
}
@Transactional
public void revoke(Long grantId) {
grantRepository.deleteById(grantId);
}
public List<ResourceGrant> listGrants(ResourceType type, String resourceId) {
return grantRepository.findByResourceTypeAndResourceId(type, normalize(resourceId));
}
/** Resource ids of the given type that this user (or their team) holds any grant on. */
public Set<String> grantedResourceIds(ResourceType type, User user) {
if (user == null) {
return Set.of();
}
Set<String> ids = new HashSet<>();
for (ResourceGrant g :
grantRepository.findByResourceTypeAndPrincipalTypeAndPrincipalId(
type, PrincipalType.USER, user.getId())) {
ids.add(g.getResourceId());
}
if (user.getTeam() != null) {
for (ResourceGrant g :
grantRepository.findByResourceTypeAndPrincipalTypeAndPrincipalId(
type, PrincipalType.TEAM, user.getTeam().getId())) {
ids.add(g.getResourceId());
}
}
return ids;
}
// ---- internals ----
private boolean hasGrant(
ResourceType type, String resourceId, User user, AccessPermission required) {
Long teamId = user.getTeam() != null ? user.getTeam().getId() : null;
for (ResourceGrant g : grantRepository.findByResourceTypeAndResourceId(type, resourceId)) {
if (!permissionSatisfies(g.getPermission(), required)) {
continue;
}
if (g.getPrincipalType() == PrincipalType.USER
&& g.getPrincipalId().equals(user.getId())) {
return true;
}
if (g.getPrincipalType() == PrincipalType.TEAM
&& teamId != null
&& g.getPrincipalId().equals(teamId)) {
return true;
}
}
return false;
}
// MANAGE implies USE.
private boolean permissionSatisfies(AccessPermission held, AccessPermission required) {
if (required == AccessPermission.USE) {
return held == AccessPermission.USE || held == AccessPermission.MANAGE;
}
return held == AccessPermission.MANAGE;
}
private boolean matchesDefault(DefaultAccessPolicy policy, User user) {
if (policy == null) {
return false;
}
return switch (policy) {
case ORG_ALL -> true;
// Admins already pass above; only team leads here.
case ADMINS_AND_TEAM_LEADS -> teamLeadLookup.isAnyTeamLeader(user);
case EXPLICIT_ONLY -> false;
};
}
private boolean isOwner(Long ownerUserId, User user) {
return ownerUserId != null && ownerUserId.equals(user.getId());
}
private boolean isAdmin(User user) {
return user.getAuthorities().stream()
.anyMatch(a -> Role.ADMIN.getRoleId().equals(a.getAuthority()));
}
private String normalize(String resourceId) {
return resourceId == null ? "" : resourceId;
}
}
@@ -1,151 +0,0 @@
package stirling.software.proprietary.access.service;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import org.springframework.stereotype.Component;
/** Masks, merges and sanitizes secret values in a config map, recursing into nested maps/lists. */
@Component
public class SecretMasker {
public static final String MASK = "********";
// Cap recursion so a pathologically nested payload cannot overflow the stack.
private static final int MAX_DEPTH = 32;
private static final Set<String> SENSITIVE_HINTS =
Set.of(
"secret",
"password",
"token",
"apikey",
"accesskey",
"credential",
"privatekey");
/** Replace sensitive values with the mask (recursively) for safe display. */
public Map<String, Object> mask(Map<String, Object> config) {
return mask(config, 0);
}
/** Drop sensitive blank/masked values from an incoming create payload. */
public Map<String, Object> sanitize(Map<String, Object> config) {
return sanitize(config, 0);
}
/**
* Merge an update over the stored map, keeping stored secrets where the incoming is redacted.
*/
public Map<String, Object> merge(Map<String, Object> stored, Map<String, Object> incoming) {
return merge(stored, incoming, 0);
}
private Map<String, Object> mask(Map<String, Object> config, int depth) {
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : config.entrySet()) {
out.put(e.getKey(), maskValue(e.getKey(), e.getValue(), depth));
}
return out;
}
private Map<String, Object> sanitize(Map<String, Object> config, int depth) {
if (config == null) {
return new LinkedHashMap<>();
}
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : config.entrySet()) {
if (isSensitive(e.getKey()) && isRedacted(e.getValue(), depth)) {
continue;
}
out.put(
e.getKey(),
e.getValue() instanceof Map<?, ?> m && depth < MAX_DEPTH
? sanitize(castMap(m), depth + 1)
: e.getValue());
}
return out;
}
private Map<String, Object> merge(
Map<String, Object> stored, Map<String, Object> incoming, int depth) {
Map<String, Object> out = new LinkedHashMap<>(stored);
for (Map.Entry<String, Object> e : incoming.entrySet()) {
String key = e.getKey();
Object value = e.getValue();
if (isSensitive(key)) {
if (!isRedacted(value, depth)) {
out.put(key, value); // a real new secret replaces the stored one
}
continue; // redacted (blank / mask) -> keep stored
}
if (depth < MAX_DEPTH
&& out.get(key) instanceof Map<?, ?> s
&& value instanceof Map<?, ?> i) {
out.put(key, merge(castMap(s), castMap(i), depth + 1));
} else {
out.put(key, value);
}
}
return out;
}
// A sensitive key masks its whole value; recurse into non-sensitive containers.
private Object maskValue(String key, Object value, int depth) {
if (isSensitive(key)) {
if (value == null || (value instanceof String s && s.isBlank())) {
return value;
}
return MASK;
}
if (depth >= MAX_DEPTH) {
// Too deep to descend; mask containers rather than risk leaking an unmasked secret.
return value instanceof Map<?, ?> || value instanceof List<?> ? MASK : value;
}
if (value instanceof Map<?, ?> m) {
return mask(castMap(m), depth + 1);
}
if (value instanceof List<?> list) {
List<Object> out = new ArrayList<>();
for (Object item : list) {
out.add(item instanceof Map<?, ?> m ? mask(castMap(m), depth + 1) : item);
}
return out;
}
return value;
}
private boolean isSensitive(String key) {
String lower = key.toLowerCase(Locale.ROOT);
return SENSITIVE_HINTS.stream().anyMatch(lower::contains);
}
/** Blank, the mask placeholder, or any structure that still contains the mask. */
private boolean isRedacted(Object value, int depth) {
if (value == null) {
return true;
}
if (value instanceof String s) {
return s.isBlank() || MASK.equals(s);
}
if (depth >= MAX_DEPTH) {
return false;
}
if (value instanceof Map<?, ?> m) {
return m.values().stream().anyMatch(v -> isRedacted(v, depth + 1));
}
if (value instanceof List<?> list) {
return list.stream().anyMatch(v -> isRedacted(v, depth + 1));
}
return false;
}
@SuppressWarnings("unchecked")
private Map<String, Object> castMap(Map<?, ?> map) {
return (Map<String, Object>) map;
}
}
@@ -1,13 +0,0 @@
package stirling.software.proprietary.access.service;
import stirling.software.proprietary.security.model.User;
/** Resolves whether a user leads a team. */
public interface TeamLeadLookup {
/** Whether the user leads at least one team. */
boolean isAnyTeamLeader(User user);
/** Whether the user leads the given team. */
boolean isLeaderOfTeam(User user, Long teamId);
}
@@ -1,116 +0,0 @@
package stirling.software.proprietary.classification;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
import stirling.software.proprietary.classification.model.TaxonomyValidator;
import stirling.software.proprietary.classification.store.TaxonomyStore;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
/**
* Read/write the caller's team classification taxonomy — the vocabulary the document classifier
* runs against. Team-scoped exactly like policies: every user reads their own team's taxonomy, and
* only a user who may edit policies (a team leader on SaaS, the global admin self-hosted; see
* {@link PolicyManagementAuthority}) may change it. Editing is gated only when login is enabled;
* single-user deployments trust the local operator. A team with no stored taxonomy reads as {@code
* 204} and the classifier falls back to the engine's built-in default.
*/
@RestController
@RequestMapping("/api/v1/classification/taxonomy")
@Hidden
@RequiredArgsConstructor
@Tag(name = "Classification", description = "Team-scoped document-classification taxonomy")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class TaxonomyController {
private final TaxonomyStore taxonomyStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ApplicationProperties applicationProperties;
private final UserServiceInterface userService;
@GetMapping
@Operation(
summary = "Get the team's classification taxonomy",
description =
"Returns the caller's team taxonomy, or 204 when the team has none (the"
+ " classifier then uses the built-in default).")
public ResponseEntity<ClassificationTaxonomy> getTaxonomy() {
return taxonomyStore
.findByTeam(currentTeamId())
.map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.noContent().build());
}
@PutMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Save the team's classification taxonomy",
description =
"Validates and stores the taxonomy for the caller's team, shared by everyone on"
+ " the team. Requires the policy-editor role for the team.")
public ResponseEntity<ClassificationTaxonomy> saveTaxonomy(
@RequestBody ClassificationTaxonomy taxonomy) {
requireEditingAllowed();
try {
TaxonomyValidator.validate(taxonomy);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
ClassificationTaxonomy saved =
taxonomyStore.save(currentTeamId(), taxonomy, currentUsername());
return ResponseEntity.ok(saved);
}
@DeleteMapping
@Operation(
summary = "Reset the team's classification taxonomy",
description =
"Removes the team's stored taxonomy so the classifier falls back to the built-in"
+ " default. Requires the policy-editor role for the team.")
public ResponseEntity<Void> resetTaxonomy() {
requireEditingAllowed();
taxonomyStore.deleteByTeam(currentTeamId());
return ResponseEntity.noContent().build();
}
/**
* Editing the taxonomy requires the editor role for the caller's team — the same gate policies
* use (team leader on SaaS, global admin self-hosted). Single-user deployments (login disabled)
* have no such role, so they trust the local operator.
*/
private void requireEditingAllowed() {
if (!applicationProperties.getSecurity().isEnableLogin()) {
return;
}
if (!policyManagementAuthority.canEditPolicies()) {
throw new ResponseStatusException(
HttpStatus.FORBIDDEN,
"The classification taxonomy may only be changed by a team leader");
}
}
private Long currentTeamId() {
return policyManagementAuthority.currentUserTeamId();
}
private String currentUsername() {
return userService == null ? null : userService.getCurrentUsername();
}
}
@@ -1,18 +0,0 @@
package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* The vocabulary a document is classified against — team-scoped and admin-editable. Its shape
* mirrors the engine's {@code ClassificationTaxonomy} contract (categories owning doc_types, plus
* free-standing cross-cutting tags), so a stored taxonomy is passed to the engine verbatim as the
* per-request override. When a team has no stored taxonomy the engine falls back to its built-in
* default.
*/
public record ClassificationTaxonomy(List<TaxonomyCategory> categories, List<String> tags) {
public ClassificationTaxonomy {
categories = categories == null ? List.of() : List.copyOf(categories);
tags = tags == null ? List.of() : List.copyOf(tags);
}
}
@@ -1,15 +0,0 @@
package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* A structural family of documents, owning the doc_types shaped like it. {@code docTypes} is
* serialized in the engine's camelCase shape (the engine's {@code ClassificationTaxonomy} model
* aliases {@code doc_types} onto it), so a stored taxonomy passes straight through to the engine.
*/
public record TaxonomyCategory(String id, String label, List<TaxonomyDocumentType> docTypes) {
public TaxonomyCategory {
docTypes = docTypes == null ? List.of() : List.copyOf(docTypes);
}
}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.classification.model;
/**
* A specific instrument within a category (e.g. {@code nda} under {@code contract}).
* Category-scoped: the engine enforces that a doc_type can only apply to its owning category.
*/
public record TaxonomyDocumentType(String id, String label) {}
@@ -1,87 +0,0 @@
package stirling.software.proprietary.classification.model;
import java.util.HashSet;
import java.util.Set;
/**
* Structural validation for an admin-supplied (or imported) taxonomy, run before it is stored so a
* malformed vocabulary can never reach the classifier. Mirrors the invariants the engine relies on:
* at least one category, non-blank ids/labels everywhere, ids unique among categories and among the
* doc_types within a category, and non-blank unique tags.
*/
public final class TaxonomyValidator {
private TaxonomyValidator() {}
// Generous upper bounds so a legitimate taxonomy is never blocked, but a single team can't
// store an unbounded blob that would bloat the row, balloon the classifier prompt, or exhaust
// memory on deserialize.
static final int MAX_CATEGORIES = 200;
static final int MAX_DOC_TYPES_PER_CATEGORY = 200;
static final int MAX_TAGS = 500;
static final int MAX_TEXT_LENGTH = 128;
/**
* @throws IllegalArgumentException with a human-readable message when the taxonomy is invalid.
*/
public static void validate(ClassificationTaxonomy taxonomy) {
if (taxonomy == null) {
throw new IllegalArgumentException("Taxonomy is required");
}
if (taxonomy.categories().isEmpty()) {
throw new IllegalArgumentException("Taxonomy must have at least one category");
}
if (taxonomy.categories().size() > MAX_CATEGORIES) {
throw new IllegalArgumentException("Too many categories (max " + MAX_CATEGORIES + ")");
}
if (taxonomy.tags().size() > MAX_TAGS) {
throw new IllegalArgumentException("Too many tags (max " + MAX_TAGS + ")");
}
Set<String> categoryIds = new HashSet<>();
for (TaxonomyCategory category : taxonomy.categories()) {
requireText(category.id(), "Category id");
requireText(category.label(), "Category label");
if (category.docTypes().size() > MAX_DOC_TYPES_PER_CATEGORY) {
throw new IllegalArgumentException(
"Too many sub-categories in '"
+ category.id()
+ "' (max "
+ MAX_DOC_TYPES_PER_CATEGORY
+ ")");
}
if (!categoryIds.add(category.id())) {
throw new IllegalArgumentException("Duplicate category id: " + category.id());
}
Set<String> docTypeIds = new HashSet<>();
for (TaxonomyDocumentType docType : category.docTypes()) {
requireText(docType.id(), "Doc type id");
requireText(docType.label(), "Doc type label");
if (!docTypeIds.add(docType.id())) {
throw new IllegalArgumentException(
"Duplicate doc type id '"
+ docType.id()
+ "' in category '"
+ category.id()
+ "'");
}
}
}
Set<String> tags = new HashSet<>();
for (String tag : taxonomy.tags()) {
requireText(tag, "Tag");
if (!tags.add(tag)) {
throw new IllegalArgumentException("Duplicate tag: " + tag);
}
}
}
private static void requireText(String value, String field) {
if (value == null || value.isBlank()) {
throw new IllegalArgumentException(field + " must not be blank");
}
if (value.length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
field + " is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
}
}
@@ -1,37 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
/**
* In-memory {@link TaxonomyStore} for tests and any future no-database mode. {@link
* JpaTaxonomyStore} is the runtime bean.
*/
public class InProcessTaxonomyStore implements TaxonomyStore {
private final Map<Long, ClassificationTaxonomy> byTeam = new ConcurrentHashMap<>();
@Override
public Optional<ClassificationTaxonomy> findByTeam(Long teamId) {
return Optional.ofNullable(byTeam.get(key(teamId)));
}
@Override
public ClassificationTaxonomy save(
Long teamId, ClassificationTaxonomy taxonomy, String updatedBy) {
byTeam.put(key(teamId), taxonomy);
return taxonomy;
}
@Override
public boolean deleteByTeam(Long teamId) {
return byTeam.remove(key(teamId)) != null;
}
private static long key(Long teamId) {
return teamId == null ? TaxonomyEntity.NO_TEAM : teamId;
}
}
@@ -1,79 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.time.Instant;
import java.util.Optional;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Durable {@link TaxonomyStore} backed by JPA; the runtime store. Gated on {@code policies.enabled}
* — a team taxonomy only matters when the Classification policy can run — so it shares the policy
* subsystem's on/off switch. The taxonomy is persisted as JSON via {@link TaxonomyEntity}.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaTaxonomyStore implements TaxonomyStore {
private final TaxonomyRepository repository;
private final ObjectMapper objectMapper;
@Override
public Optional<ClassificationTaxonomy> findByTeam(Long teamId) {
Optional<TaxonomyEntity> entity = repository.findById(key(teamId));
if (entity.isEmpty()) {
return Optional.empty();
}
try {
return Optional.of(
objectMapper.readValue(
entity.get().getTaxonomyJson(), ClassificationTaxonomy.class));
} catch (JacksonException e) {
// A stored taxonomy that no longer parses (corruption / manual DB edit) must not break
// classification: drop it so the caller falls back to the built-in default rather than
// surfacing a 500 on every upload for the team.
log.warn(
"Discarding unparseable stored taxonomy for team {}: {}",
teamId,
e.getMessage());
return Optional.empty();
}
}
@Override
public ClassificationTaxonomy save(
Long teamId, ClassificationTaxonomy taxonomy, String updatedBy) {
TaxonomyEntity entity = new TaxonomyEntity();
entity.setTeamId(key(teamId));
entity.setTaxonomyJson(objectMapper.writeValueAsString(taxonomy));
entity.setUpdatedAt(Instant.now());
entity.setUpdatedBy(updatedBy);
repository.save(entity);
return taxonomy;
}
@Override
public boolean deleteByTeam(Long teamId) {
long id = key(teamId);
if (!repository.existsById(id)) {
return false;
}
repository.deleteById(id);
return true;
}
/** Map the nullable team id onto the entity's non-null key (sentinel for the unteamed case). */
private static long key(Long teamId) {
return teamId == null ? TaxonomyEntity.NO_TEAM : teamId;
}
}
@@ -1,47 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.io.Serializable;
import java.time.Instant;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* JPA row for a team's classification taxonomy — one row per team. The taxonomy lives as JSON in
* {@code taxonomyJson} (authoritative on read). {@code teamId} is the natural key; the sentinel
* {@link #NO_TEAM} stands in for the unteamed (login-disabled / self-hosted single-team) case,
* since a primary key can't be null (policies store a nullable {@code team_id}, but this table is
* keyed one-per-team). Kept decoupled from the security entities — {@code teamId} is a plain value,
* not a foreign key — so classification can be enabled or disabled without touching them.
*/
@Entity
@Table(name = "classification_taxonomies")
@NoArgsConstructor
@Getter
@Setter
public class TaxonomyEntity implements Serializable {
private static final long serialVersionUID = 1L;
/** Sentinel key for the unteamed taxonomy (login disabled / no resolvable team). */
public static final long NO_TEAM = 0L;
@Id
@Column(name = "team_id")
private long teamId;
@Column(name = "taxonomy_json", columnDefinition = "text")
private String taxonomyJson;
@Column(name = "updated_at")
private Instant updatedAt;
@Column(name = "updated_by")
private String updatedBy;
}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.classification.store;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface TaxonomyRepository extends JpaRepository<TaxonomyEntity, Long> {}
@@ -1,22 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Optional;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
/**
* Stores one {@link ClassificationTaxonomy} per team. A {@code null} teamId addresses the unteamed
* taxonomy (login disabled / no resolvable team), mirroring how the policy store treats a null
* team.
*/
public interface TaxonomyStore {
/** The team's stored taxonomy, or empty when it has none (callers fall back to the default). */
Optional<ClassificationTaxonomy> findByTeam(Long teamId);
/** Create or replace the team's taxonomy. Returns the stored value. */
ClassificationTaxonomy save(Long teamId, ClassificationTaxonomy taxonomy, String updatedBy);
/** Remove the team's taxonomy (reset to default). Returns whether one existed. */
boolean deleteByTeam(Long teamId);
}
@@ -1,193 +0,0 @@
package stirling.software.proprietary.controller.api;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.classification.store.TaxonomyStore;
import stirling.software.proprietary.model.api.ai.AiPageText;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.PdfContentExtractor;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.node.ObjectNode;
/**
* Dispatchable tool that classifies a PDF and writes the result into its metadata.
*
* <p>Runs as a Classification-policy pipeline step: it reads a bounded page window, asks the AI
* engine to classify the document, and stores the engine's JSON answer — minus the transport-only
* {@code outcome} field — in the custom Info-dictionary key {@link
* PdfMetadataService#CLASSIFICATION_KEY}. Returns the tagged PDF. Not intended for direct client
* use.
*/
@Slf4j
@Hidden
@RestController
@RequestMapping("/api/v1/ai/tools")
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
public class ClassifyTagController {
/** Pages read from each end of the document — mirrors the engine's window. */
private static final int WINDOW_PAGES = 2;
private static final String CLASSIFY_ENDPOINT = "/api/v1/documents/classify";
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private final PdfContentExtractor pdfContentExtractor;
private final PdfMetadataService pdfMetadataService;
private final AiEngineClient aiEngineClient;
private final ObjectMapper objectMapper;
private final UserServiceInterface userService;
/**
* Present only when the policy subsystem is enabled ({@code policies.enabled}); the store and
* team authority are gated on it. Null otherwise, in which case classification falls back to
* the engine's built-in default taxonomy.
*/
private final TaxonomyStore taxonomyStore;
private final PolicyManagementAuthority policyManagementAuthority;
public ClassifyTagController(
CustomPDFDocumentFactory pdfDocumentFactory,
TempFileManager tempFileManager,
PdfContentExtractor pdfContentExtractor,
PdfMetadataService pdfMetadataService,
AiEngineClient aiEngineClient,
ObjectMapper objectMapper,
@Autowired(required = false) UserServiceInterface userService,
@Autowired(required = false) TaxonomyStore taxonomyStore,
@Autowired(required = false) PolicyManagementAuthority policyManagementAuthority) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.tempFileManager = tempFileManager;
this.pdfContentExtractor = pdfContentExtractor;
this.pdfMetadataService = pdfMetadataService;
this.aiEngineClient = aiEngineClient;
this.objectMapper = objectMapper;
this.userService = userService;
this.taxonomyStore = taxonomyStore;
this.policyManagementAuthority = policyManagementAuthority;
}
@PostMapping(value = "/classify-and-tag", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Classify a PDF and tag its metadata",
description =
"Reads the first two and last two pages, classifies the document via the AI"
+ " engine, and stores the result in the StirlingPDFClassification"
+ " metadata field. Dispatched by the Classification policy; not"
+ " intended for direct client use.")
public ResponseEntity<Resource> classifyAndTag(
@RequestParam("fileInput") MultipartFile fileInput) throws IOException {
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
String fileName = safeFileName(fileInput.getOriginalFilename());
List<AiPageText> pages = extractWindow(document);
String requestBody =
objectMapper.writeValueAsString(
new ClassifyEngineRequest(fileName, pages, resolveTaxonomyOverride()));
String userId = userService != null ? userService.getCurrentUsername() : null;
String responseJson = aiEngineClient.post(CLASSIFY_ENDPOINT, requestBody, userId);
pdfMetadataService.setClassificationMetadata(document, toMetadataValue(responseJson));
log.debug("[classify-and-tag] tagged {} ({} window pages)", fileName, pages.size());
return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
}
}
private List<AiPageText> extractWindow(PDDocument document) throws IOException {
List<AiPageText> pages = new ArrayList<>();
for (int pageNumber : windowPageNumbers(document.getNumberOfPages(), WINDOW_PAGES)) {
String text = pdfContentExtractor.extractPageTextRaw(document, pageNumber);
if (text != null && !text.isBlank()) {
pages.add(new AiPageText(pageNumber, text));
}
}
return pages;
}
/** First and last {@code window} page numbers (1-based), de-duplicated and in order. */
static List<Integer> windowPageNumbers(int pageCount, int window) {
Set<Integer> numbers = new LinkedHashSet<>();
for (int page = 1; page <= Math.min(window, pageCount); page++) {
numbers.add(page);
}
for (int page = Math.max(1, pageCount - window + 1); page <= pageCount; page++) {
numbers.add(page);
}
return new ArrayList<>(numbers);
}
/** Drop the transport-only {@code outcome} discriminator; keep the rest verbatim. */
private String toMetadataValue(String engineResponseJson) {
JsonNode node = objectMapper.readTree(engineResponseJson);
if (node instanceof ObjectNode object) {
object.remove("outcome");
}
return objectMapper.writeValueAsString(node);
}
private static String safeFileName(String originalFilename) {
String name = Filenames.toSimpleFileName(originalFilename);
return (name == null || name.isBlank()) ? "classified.pdf" : name;
}
/**
* Resolve the caller's team taxonomy and return it in the engine's shape to classify against;
* {@code null} falls back to the engine's generated default. The stored taxonomy is already in
* the engine's camelCase shape ({@code categories}/{@code docTypes}/{@code tags}), so it is
* passed through verbatim. Returns null when the policy subsystem is disabled (no store), when
* the team has no stored taxonomy, or when the team can't be resolved.
*/
private JsonNode resolveTaxonomyOverride() {
if (taxonomyStore == null) {
return null;
}
Long teamId =
policyManagementAuthority == null
? null
: policyManagementAuthority.currentUserTeamId();
return taxonomyStore
.findByTeam(teamId)
.map(taxonomy -> (JsonNode) objectMapper.valueToTree(taxonomy))
.orElse(null);
}
/** Request body for the engine's {@code /api/v1/documents/classify} endpoint. */
@JsonInclude(JsonInclude.Include.NON_NULL)
private record ClassifyEngineRequest(
String fileName, List<AiPageText> pages, JsonNode taxonomy) {}
}
@@ -1,81 +0,0 @@
package stirling.software.proprietary.integration.controller;
import java.util.List;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.integration.dto.IntegrationConfigRequest;
import stirling.software.proprietary.integration.dto.IntegrationConfigResponse;
import stirling.software.proprietary.integration.service.IntegrationConfigService;
import stirling.software.proprietary.security.model.User;
/** CRUD for S3/MCP/API integration configs. Secrets are never returned. */
@RestController
@RequestMapping("/api/v1/integrations")
@RequiredArgsConstructor
@PreAuthorize("isAuthenticated()")
@Tag(name = "Integrations", description = "Manage S3/MCP/API integration configurations")
public class IntegrationConfigController {
private final IntegrationConfigService service;
@GetMapping
public ResponseEntity<List<IntegrationConfigResponse>> list(
@AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(
service.listVisible(user).stream().map(c -> service.toResponse(c, user)).toList());
}
@PostMapping
public ResponseEntity<IntegrationConfigResponse> create(
@RequestBody IntegrationConfigRequest request, @AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.create(request, user), user));
}
@GetMapping("/{id}")
public ResponseEntity<IntegrationConfigResponse> get(
@PathVariable Long id, @AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.getForUse(id, user), user));
}
@PutMapping("/{id}")
public ResponseEntity<IntegrationConfigResponse> update(
@PathVariable Long id,
@RequestBody IntegrationConfigRequest request,
@AuthenticationPrincipal User user) {
requireUser(user);
return ResponseEntity.ok(service.toResponse(service.update(id, request, user), user));
}
@DeleteMapping("/{id}")
public ResponseEntity<Void> delete(@PathVariable Long id, @AuthenticationPrincipal User user) {
requireUser(user);
service.delete(id, user);
return ResponseEntity.noContent().build();
}
private void requireUser(User user) {
if (user == null) {
throw new ResponseStatusException(HttpStatus.UNAUTHORIZED, "Authentication required");
}
}
}
@@ -1,135 +0,0 @@
package stirling.software.proprietary.integration.crypto;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.GeneralSecurityException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import jakarta.annotation.PostConstruct;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.InstallationPathConfig;
/**
* AES-256-GCM for stored credentials. Key from property, env var, or an auto-generated key file.
*/
@Component
@Slf4j
public class CredentialEncryption {
private static final String ALGORITHM = "AES";
private static final String TRANSFORMATION = "AES/GCM/NoPadding";
private static final int GCM_TAG_BITS = 128;
private static final int IV_BYTES = 12;
private static final String KEY_FILE = "credential-encryption.key";
private static final SecureRandom RANDOM = new SecureRandom();
private static volatile SecretKey key;
private final String configuredKey;
public CredentialEncryption(
@Value("${stirling.security.credentialEncryptionKey:}") String configuredKey) {
this.configuredKey = configuredKey;
}
@PostConstruct
void init() {
key = resolveKey();
log.info("Credential encryption initialised (AES-256-GCM)");
}
private SecretKey resolveKey() {
String configured = configuredKey;
if (configured == null || configured.isBlank()) {
configured = System.getenv("STIRLING_CREDENTIAL_ENCRYPTION_KEY");
}
if (configured != null && !configured.isBlank()) {
return new SecretKeySpec(Base64.getDecoder().decode(configured.trim()), ALGORITHM);
}
return loadOrCreateKeyFile();
}
private SecretKey loadOrCreateKeyFile() {
Path path = Path.of(InstallationPathConfig.getConfigPath(), KEY_FILE);
try {
if (Files.exists(path)) {
String encoded = Files.readString(path).trim();
return new SecretKeySpec(Base64.getDecoder().decode(encoded), ALGORITHM);
}
KeyGenerator generator = KeyGenerator.getInstance(ALGORITHM);
generator.init(256);
SecretKey generated = generator.generateKey();
Files.createDirectories(path.getParent());
Files.writeString(path, Base64.getEncoder().encodeToString(generated.getEncoded()));
log.warn(
"Generated a new credential encryption key at {}. Back this file up: losing it"
+ " makes stored integration secrets unrecoverable.",
path);
return generated;
} catch (Exception e) {
throw new IllegalStateException("Unable to initialise credential encryption key", e);
}
}
public static String encrypt(String plaintext) {
if (plaintext == null) {
return null;
}
try {
byte[] iv = new byte[IV_BYTES];
RANDOM.nextBytes(iv);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.ENCRYPT_MODE, requireKey(), new GCMParameterSpec(GCM_TAG_BITS, iv));
byte[] ciphertext = cipher.doFinal(plaintext.getBytes(StandardCharsets.UTF_8));
byte[] combined = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, combined, 0, iv.length);
System.arraycopy(ciphertext, 0, combined, iv.length, ciphertext.length);
return Base64.getEncoder().encodeToString(combined);
} catch (GeneralSecurityException e) {
throw new IllegalStateException("Failed to encrypt credential", e);
}
}
public static String decrypt(String stored) {
if (stored == null) {
return null;
}
try {
byte[] combined = Base64.getDecoder().decode(stored);
byte[] iv = Arrays.copyOfRange(combined, 0, IV_BYTES);
byte[] ciphertext = Arrays.copyOfRange(combined, IV_BYTES, combined.length);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.DECRYPT_MODE, requireKey(), new GCMParameterSpec(GCM_TAG_BITS, iv));
return new String(cipher.doFinal(ciphertext), StandardCharsets.UTF_8);
} catch (GeneralSecurityException e) {
throw new IllegalStateException("Failed to decrypt credential", e);
}
}
private static SecretKey requireKey() {
SecretKey current = key;
if (current == null) {
throw new IllegalStateException("Credential encryption not initialised");
}
return current;
}
/** For tests. */
static void initialiseForTesting(SecretKey testKey) {
key = testKey;
}
}
@@ -1,19 +0,0 @@
package stirling.software.proprietary.integration.crypto;
import jakarta.persistence.AttributeConverter;
import jakarta.persistence.Converter;
/** Transparently encrypts/decrypts a string column at rest via {@link CredentialEncryption}. */
@Converter
public class EncryptedStringConverter implements AttributeConverter<String, String> {
@Override
public String convertToDatabaseColumn(String attribute) {
return CredentialEncryption.encrypt(attribute);
}
@Override
public String convertToEntityAttribute(String dbData) {
return CredentialEncryption.decrypt(dbData);
}
}
@@ -1,18 +0,0 @@
package stirling.software.proprietary.integration.dto;
import java.util.Map;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationType;
/** Create/update payload for an integration config. Sensitive config values left blank are kept. */
public record IntegrationConfigRequest(
IntegrationType integrationType,
String name,
OwnerScope scope,
Long ownerTeamId,
Boolean enabled,
Boolean locked,
DefaultAccessPolicy defaultAccess,
Map<String, Object> config) {}
@@ -1,24 +0,0 @@
package stirling.software.proprietary.integration.dto;
import java.time.LocalDateTime;
import java.util.Map;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationType;
/** Integration config view. Sensitive config values are masked. */
public record IntegrationConfigResponse(
Long id,
IntegrationType integrationType,
String name,
OwnerScope scope,
Long ownerUserId,
Long ownerTeamId,
boolean enabled,
boolean locked,
DefaultAccessPolicy defaultAccess,
Map<String, Object> config,
boolean canManage,
LocalDateTime createdAt,
LocalDateTime updatedAt) {}
@@ -1,67 +0,0 @@
package stirling.software.proprietary.integration.model;
import java.io.Serializable;
import java.time.LocalDateTime;
import org.hibernate.annotations.CreationTimestamp;
import org.hibernate.annotations.UpdateTimestamp;
import jakarta.persistence.Column;
import jakarta.persistence.Convert;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Index;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.integration.crypto.EncryptedStringConverter;
/** A named S3/MCP/API integration config; scope and ownership live on {@link OwnedResource}. */
@Entity
@Table(
name = "integration_configs",
indexes = {
@Index(name = "idx_integration_configs_owner", columnList = "owner_user_id"),
@Index(name = "idx_integration_configs_type", columnList = "integration_type"),
@Index(name = "idx_integration_configs_scope", columnList = "scope")
})
@NoArgsConstructor
@Getter
@Setter
public class IntegrationConfig extends OwnedResource implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "integration_config_id")
private Long id;
@Enumerated(EnumType.STRING)
@Column(name = "integration_type", nullable = false, length = 32)
private IntegrationType integrationType;
@Column(name = "name", nullable = false)
private String name;
// Type-specific fields as an AES-GCM encrypted JSON blob.
@Convert(converter = EncryptedStringConverter.class)
@Column(name = "config_encrypted", columnDefinition = "text")
private String config;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
@UpdateTimestamp
@Column(name = "updated_at")
private LocalDateTime updatedAt;
}
@@ -1,8 +0,0 @@
package stirling.software.proprietary.integration.model;
/** Kind of external integration a stored config describes. */
public enum IntegrationType {
S3,
MCP,
API
}
@@ -1,21 +0,0 @@
package stirling.software.proprietary.integration.repository;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.User;
@Repository
public interface IntegrationConfigRepository extends JpaRepository<IntegrationConfig, Long> {
List<IntegrationConfig> findByOwnerUser(User ownerUser);
List<IntegrationConfig> findByOwnerTeam(Team ownerTeam);
List<IntegrationConfig> findByScope(OwnerScope scope);
}
@@ -1,226 +0,0 @@
package stirling.software.proprietary.integration.service;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.server.ResponseStatusException;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.access.service.SecretMasker;
import stirling.software.proprietary.integration.dto.IntegrationConfigRequest;
import stirling.software.proprietary.integration.dto.IntegrationConfigResponse;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.ObjectMapper;
/** CRUD for {@link IntegrationConfig}; delegates ownership and masking to shared services. */
@Service
@RequiredArgsConstructor
@Slf4j
@Transactional(readOnly = true)
public class IntegrationConfigService {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private final IntegrationConfigRepository repository;
private final OwnershipService ownership;
private final SecretMasker secretMasker;
// ---- commands ----
@Transactional
public IntegrationConfig create(IntegrationConfigRequest request, User currentUser) {
OwnerScope scope = request.scope() == null ? OwnerScope.USER : request.scope();
IntegrationConfig cfg = new IntegrationConfig();
cfg.setIntegrationType(require(request.integrationType(), "integrationType"));
cfg.setName(require(request.name(), "name"));
cfg.setEnabled(request.enabled() == null || request.enabled());
cfg.setLocked(request.locked() != null && request.locked());
cfg.setDefaultAccess(
request.defaultAccess() == null
? DefaultAccessPolicy.EXPLICIT_ONLY
: request.defaultAccess());
ownership.assignOwnership(
cfg,
scope,
request.ownerTeamId(),
currentUser,
() -> lockedServerExists(cfg.getIntegrationType()));
cfg.setConfig(writeJson(secretMasker.sanitize(request.config())));
return repository.save(cfg);
}
@Transactional
public IntegrationConfig update(Long id, IntegrationConfigRequest request, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canManage(TYPE, cfg, currentUser)) {
throw forbidden("You cannot manage this integration");
}
if (cfg.isLocked() && !ownership.isAdmin(currentUser)) {
throw forbidden("This integration is locked by an administrator");
}
if (request.name() != null) {
cfg.setName(request.name());
}
if (request.enabled() != null) {
cfg.setEnabled(request.enabled());
}
if (request.locked() != null && request.locked() != cfg.isLocked()) {
if (!ownership.isAdmin(currentUser)) {
throw forbidden("Only administrators can change the locked flag");
}
cfg.setLocked(request.locked());
}
if (request.defaultAccess() != null) {
cfg.setDefaultAccess(request.defaultAccess());
}
if (request.config() != null) {
cfg.setConfig(
writeJson(secretMasker.merge(readJson(cfg.getConfig()), request.config())));
}
return repository.save(cfg);
}
@Transactional
public void delete(Long id, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canManage(TYPE, cfg, currentUser)) {
throw forbidden("You cannot manage this integration");
}
repository.delete(cfg);
}
// ---- queries ----
public IntegrationConfig getForUse(Long id, User currentUser) {
IntegrationConfig cfg = load(id);
if (!ownership.canUse(TYPE, cfg, currentUser)) {
throw forbidden("You cannot access this integration");
}
return cfg;
}
/**
* All configs the user owns, plus server/team configs and grant-shared configs they may use.
*/
public List<IntegrationConfig> listVisible(User currentUser) {
Map<Long, IntegrationConfig> byId = new LinkedHashMap<>();
for (IntegrationConfig c : repository.findByOwnerUser(currentUser)) {
byId.put(c.getId(), c);
}
for (IntegrationConfig c : repository.findByScope(OwnerScope.SERVER)) {
if (ownership.canUse(TYPE, c, currentUser)) {
byId.put(c.getId(), c);
}
}
if (currentUser.getTeam() != null) {
for (IntegrationConfig c : repository.findByOwnerTeam(currentUser.getTeam())) {
if (ownership.canUse(TYPE, c, currentUser)) {
byId.put(c.getId(), c);
}
}
}
for (String rid : ownership.grantedResourceIds(TYPE, currentUser)) {
if (rid == null || rid.isBlank()) {
continue;
}
Long cid;
try {
cid = Long.valueOf(rid);
} catch (NumberFormatException e) {
continue;
}
if (byId.containsKey(cid)) {
continue;
}
repository
.findById(cid)
.filter(c -> ownership.canUse(TYPE, c, currentUser))
.ifPresent(c -> byId.put(c.getId(), c));
}
return new ArrayList<>(byId.values());
}
public IntegrationConfigResponse toResponse(IntegrationConfig cfg, User user) {
return new IntegrationConfigResponse(
cfg.getId(),
cfg.getIntegrationType(),
cfg.getName(),
cfg.getScope(),
cfg.getOwnerUserId(),
cfg.getOwnerTeamId(),
cfg.isEnabled(),
cfg.isLocked(),
cfg.getDefaultAccess(),
secretMasker.mask(readJson(cfg.getConfig())),
ownership.canManage(TYPE, cfg, user),
cfg.getCreatedAt(),
cfg.getUpdatedAt());
}
// ---- integration-specific glue ----
/** A non-admin can't create a personal config of a type an admin has locked at server scope. */
private boolean lockedServerExists(IntegrationType type) {
return repository.findByScope(OwnerScope.SERVER).stream()
.anyMatch(c -> c.getIntegrationType() == type && c.isLocked());
}
private IntegrationConfig load(Long id) {
return repository
.findById(id)
.orElseThrow(
() ->
new ResponseStatusException(
HttpStatus.NOT_FOUND, "Integration not found"));
}
private String writeJson(Map<String, Object> config) {
try {
return OBJECT_MAPPER.writeValueAsString(config == null ? Map.of() : config);
} catch (Exception e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Invalid config payload");
}
}
private Map<String, Object> readJson(String json) {
if (json == null || json.isBlank()) {
return new LinkedHashMap<>();
}
try {
return OBJECT_MAPPER.readValue(
json, new TypeReference<LinkedHashMap<String, Object>>() {});
} catch (Exception e) {
log.error("Failed to parse integration config JSON", e);
return new LinkedHashMap<>();
}
}
private <T> T require(T value, String field) {
if (value == null || (value instanceof String s && s.isBlank())) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, field + " is required");
}
return value;
}
private ResponseStatusException forbidden(String message) {
return new ResponseStatusException(HttpStatus.FORBIDDEN, message);
}
}
@@ -21,11 +21,4 @@ public class AiWorkflowResultFile {
@Schema(description = "MIME type of the file", example = "application/pdf")
private String contentType;
@Schema(
description =
"Index into the request's fileInputs that this output was derived from, or null"
+ " when it has no single source (e.g. a merge, or a generated file)."
+ " Lets the client replace that input in place as a new version.")
private Integer sourceIndex;
}
@@ -8,11 +8,7 @@ import tools.jackson.databind.JsonNode;
/**
* Result of a {@link PolicyExecutor} run. {@code files} are final temp files (not yet stored).
* {@code origins} is parallel to {@code files}: each entry is the index into the original pipeline
* inputs that the output traces back to, or {@code null} when it has no single source (e.g. a merge
* combining several inputs, or a generated file). Callers use it to map an output back onto the
* file it came from. {@code report}/{@code reportTool} carry the last step's structured report and
* its operation, or null if no step produced one.
* {@code report}/{@code reportTool} carry the last step's structured report and its operation, or
* null if no step produced one.
*/
public record PolicyExecutionResult(
List<Resource> files, List<Integer> origins, JsonNode report, String reportTool) {}
public record PolicyExecutionResult(List<Resource> files, JsonNode report, String reportTool) {}
@@ -58,11 +58,6 @@ public class PolicyExecutor {
// payload the tool surfaced alongside or instead of a file.
private record ToolResult(List<Resource> files, JsonNode report) {}
// A step's output files paired with each file's origin (the index into the original pipeline
// inputs it traces back to, or null when it has no single source). Origins compose across steps
// so the final result can be mapped back onto the files that entered the pipeline.
private record StepOutput(List<Resource> files, List<Integer> origins, JsonNode report) {}
/**
* Run every step in order, feeding each step's output into the next. Supporting files in {@code
* inputs} bind to named file fields and never enter the document stream.
@@ -80,12 +75,6 @@ public class PolicyExecutor {
List<Resource> currentFiles = inputs.primary();
Map<String, List<Resource>> supportingFiles = inputs.supportingFiles();
// Seed each input with its own index as origin; steps carry these through so the final
// outputs can be traced back to the files that entered the pipeline.
List<Integer> currentOrigins = new ArrayList<>();
for (int k = 0; k < currentFiles.size(); k++) {
currentOrigins.add(k);
}
// Last non-null report wins: the terminal step defines the output.
JsonNode lastReport = null;
String lastReportTool = null;
@@ -98,10 +87,8 @@ public class PolicyExecutor {
"Pipeline step " + (i + 1) + " has no operation");
}
listener.onStepStart(i + 1, steps.size(), operation);
StepOutput stepResult =
executeStep(step, currentFiles, currentOrigins, supportingFiles);
ToolResult stepResult = executeStep(step, currentFiles, supportingFiles);
currentFiles = stepResult.files();
currentOrigins = stepResult.origins();
if (stepResult.report() != null) {
lastReport = stepResult.report();
lastReportTool = operation;
@@ -109,7 +96,7 @@ public class PolicyExecutor {
listener.onStepComplete(i + 1, steps.size(), operation);
}
return new PolicyExecutionResult(currentFiles, currentOrigins, lastReport, lastReportTool);
return new PolicyExecutionResult(currentFiles, lastReport, lastReportTool);
}
/**
@@ -117,50 +104,32 @@ public class PolicyExecutor {
* responses are unpacked so each inner file is its own result (e.g. split). For per-file
* dispatch the first non-null report wins.
*/
private StepOutput executeStep(
private ToolResult executeStep(
PipelineStep step,
List<Resource> inputFiles,
List<Integer> inputOrigins,
Map<String, List<Resource>> supportingFiles)
throws IOException {
requireAcceptedTypes(step.operation(), inputFiles);
List<Resource> files = new ArrayList<>();
List<Integer> origins = new ArrayList<>();
JsonNode report = null;
if (toolMetadataService.isMultiInput(step.operation())) {
// One call over all inputs. The outputs derive from a single input only when exactly
// one entered; otherwise (a genuine merge) there is no single source.
ToolResult r = callEndpoint(step, inputFiles, supportingFiles);
Integer origin = inputOrigins.size() == 1 ? inputOrigins.get(0) : null;
for (Resource file : r.files()) {
files.add(file);
origins.add(origin);
}
files.addAll(r.files());
report = r.report();
} else if (inputFiles.isEmpty()) {
ToolResult r = callEndpoint(step, List.of(), supportingFiles);
for (Resource file : r.files()) {
files.add(file);
origins.add(null);
}
files.addAll(r.files());
report = r.report();
} else {
// One call per file: every output of this call inherits that input's origin, so a 1:1
// op keeps its chain and a split (one input, many outputs) tags each output with the
// same source.
for (int k = 0; k < inputFiles.size(); k++) {
Integer origin = inputOrigins.get(k);
ToolResult r = callEndpoint(step, List.of(inputFiles.get(k)), supportingFiles);
for (Resource file : r.files()) {
files.add(file);
origins.add(origin);
}
for (Resource file : inputFiles) {
ToolResult r = callEndpoint(step, List.of(file), supportingFiles);
files.addAll(r.files());
if (report == null) {
report = r.report();
}
}
}
return new StepOutput(files, origins, report);
return new ToolResult(files, report);
}
/**
@@ -21,7 +21,6 @@ import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.model.PolicyRunStatus;
import stirling.software.proprietary.policy.progress.PolicyProgressListener;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceDocCounter;
import stirling.software.proprietary.policy.source.SourceStore;
/**
@@ -38,7 +37,6 @@ public class PolicyRunner {
private final PolicyEngine policyEngine;
private final List<InputSource> inputSources;
private final SourceStore sourceStore;
private final SourceDocCounter docCounter;
/**
* Trigger entry point. Pulls every referenced source; each yielded unit becomes its own run so
@@ -67,7 +65,7 @@ public class PolicyRunner {
policy.id());
continue;
}
runIds.addAll(pullAndRun(policy, sourceId, source.toInputSpec()));
runIds.addAll(pullAndRun(policy, source.toInputSpec()));
}
return runIds;
}
@@ -84,11 +82,7 @@ public class PolicyRunner {
return policyEngine.submit(definition, inputs, listener);
}
/**
* Resolves the source and starts a run per unit; records how many documents the source fed and
* returns the ids of the runs started.
*/
private List<String> pullAndRun(Policy policy, String sourceId, InputSpec spec) {
private List<String> pullAndRun(Policy policy, InputSpec spec) {
InputSource source = sourceFor(spec);
if (source == null) {
log.warn(
@@ -109,12 +103,9 @@ public class PolicyRunner {
return List.of();
}
List<String> runIds = new ArrayList<>();
long docsFed = 0;
for (ResolvedInput unit : work) {
runIds.add(startRun(policy, unit.inputs(), unit.onComplete()));
docsFed += unit.inputs().primary().size();
}
docCounter.record(sourceId, docsFed);
return runIds;
}
@@ -1,15 +0,0 @@
package stirling.software.proprietary.policy.source;
/**
* Per-source document throughput for the overview row: how many documents a source has fed into
* runs in total and over the trailing 24-hour and 30-day windows. Counts documents <em>fed</em>
* (picked up by a run), so a snapshot-mode source that re-reads the same files each run counts them
* per run.
*/
public record DocStats(long total, long last24h, long last30d) {
/** Number of trailing daily buckets in a source's daily series. */
public static final int DAYS = 30;
public static final DocStats ZERO = new DocStats(0, 0, 0);
}
@@ -1,64 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Supplier;
/**
* In-memory {@link SourceDocCounter} for tests and any future no-database mode. Holds hourly
* buckets per source; the clock is injectable so window boundaries can be exercised
* deterministically. {@link JpaSourceDocCounter} is the runtime bean.
*/
public class InProcessSourceDocCounter implements SourceDocCounter {
private final Supplier<Instant> clock;
private final Map<String, Map<Long, Long>> bucketsBySource = new ConcurrentHashMap<>();
public InProcessSourceDocCounter() {
this(Instant::now);
}
public InProcessSourceDocCounter(Supplier<Instant> clock) {
this.clock = clock;
}
@Override
public void record(String sourceId, long docs) {
if (docs <= 0) {
return;
}
bucketsBySource
.computeIfAbsent(sourceId, key -> new ConcurrentHashMap<>())
.merge(currentHour(), docs, Long::sum);
}
@Override
public Map<String, DocStats> statsFor(Collection<String> sourceIds) {
long now = currentHour();
Map<String, DocStats> stats = new HashMap<>();
for (String id : sourceIds) {
Map<Long, Long> buckets = bucketsBySource.getOrDefault(id, Map.of());
long total = buckets.values().stream().mapToLong(Long::longValue).sum();
long last24h =
SourceDocWindows.sumSince(buckets, now - (SourceDocWindows.HOURS_IN_24H - 1));
long last30d = SourceDocWindows.sumSince(buckets, SourceDocWindows.firstDayHour(now));
stats.put(id, new DocStats(total, last24h, last30d));
}
return stats;
}
@Override
public List<Long> dailySeriesFor(String sourceId) {
long now = currentHour();
Map<Long, Long> buckets = bucketsBySource.getOrDefault(sourceId, Map.of());
return SourceDocWindows.series(SourceDocWindows.byDay(buckets), now / 24);
}
private long currentHour() {
return clock.get().getEpochSecond() / 3600;
}
}
@@ -1,154 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.function.IntSupplier;
import java.util.function.Supplier;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
/**
* Durable {@link SourceDocCounter}; the runtime bean. {@code record} keeps two things in step: an
* hourly bucket ({@link SourceDocCountEntity}) that feeds the rolling 24h / 30d / daily-series
* windows, and a denormalized lifetime total ({@link SourceDocTotalEntity}) read directly for the
* all-time figure. The lifetime counter means the overview never scans a source's whole bucket
* history, and lets {@link #pruneOldBuckets()} retire buckets past the 30-day window so the hourly
* table stays bounded (~one row per source per active hour, for at most 30 days).
*/
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaSourceDocCounter implements SourceDocCounter {
private final SourceDocCountRepository countRepository;
private final SourceDocTotalRepository totalRepository;
private final Supplier<Instant> clock;
@Autowired
public JpaSourceDocCounter(
SourceDocCountRepository countRepository, SourceDocTotalRepository totalRepository) {
this(countRepository, totalRepository, Instant::now);
}
// Clock seam so tests can pin "now"; the runtime bean uses the wall clock above.
JpaSourceDocCounter(
SourceDocCountRepository countRepository,
SourceDocTotalRepository totalRepository,
Supplier<Instant> clock) {
this.countRepository = countRepository;
this.totalRepository = totalRepository;
this.clock = clock;
}
@Override
public void record(String sourceId, long docs) {
if (docs <= 0) {
return;
}
long bucketHour = currentHour();
upsert(
() -> totalRepository.increment(sourceId, docs),
() -> totalRepository.saveAndFlush(new SourceDocTotalEntity(sourceId, docs)));
upsert(
() -> countRepository.increment(sourceId, bucketHour, docs),
() ->
countRepository.saveAndFlush(
new SourceDocCountEntity(sourceId, bucketHour, docs)));
}
/**
* Add {@code docs} to a per-source running total: increment the existing row, else insert a new
* one. The insert is flushed now (in its own transaction, since {@code record} is not
* {@code @Transactional}) so a concurrent run's winning insert surfaces as a constraint
* violation we retry as an increment, rather than as a silent {@code merge} overwrite or a
* later doomed commit.
*/
private static void upsert(IntSupplier increment, Runnable insert) {
if (increment.getAsInt() > 0) {
return;
}
try {
insert.run();
} catch (DataIntegrityViolationException concurrentInsert) {
increment.getAsInt();
}
}
@Override
public Map<String, DocStats> statsFor(Collection<String> sourceIds) {
if (sourceIds.isEmpty()) {
return Map.of();
}
long now = currentHour();
Map<String, Long> totals = sums(totalRepository.totalsFor(sourceIds));
Map<String, Long> last24h =
sums(
countRepository.sumBySourceSince(
sourceIds, now - (SourceDocWindows.HOURS_IN_24H - 1)));
Map<String, Long> last30d =
sums(
countRepository.sumBySourceSince(
sourceIds, SourceDocWindows.firstDayHour(now)));
Map<String, DocStats> stats = new HashMap<>();
for (String id : sourceIds) {
stats.put(
id,
new DocStats(
totals.getOrDefault(id, 0L),
last24h.getOrDefault(id, 0L),
last30d.getOrDefault(id, 0L)));
}
return stats;
}
@Override
public List<Long> dailySeriesFor(String sourceId) {
long now = currentHour();
Collection<String> ids = List.of(sourceId);
Map<Long, Long> dailyCounts =
dailyBySource(
countRepository.dailyCountsSince(
ids, SourceDocWindows.firstDayHour(now)))
.getOrDefault(sourceId, Map.of());
return SourceDocWindows.series(dailyCounts, now / 24);
}
/**
* Retire hourly buckets older than the 30-day window; the lifetime total is held separately, so
* nothing reported is lost. Daily is ample - the read queries already ignore older buckets, so
* this is purely storage hygiene.
*/
@Scheduled(fixedDelay = 1, timeUnit = TimeUnit.DAYS)
public void pruneOldBuckets() {
countRepository.deleteOlderThan(SourceDocWindows.firstDayHour(currentHour()));
}
private long currentHour() {
return clock.get().getEpochSecond() / 3600;
}
private static Map<String, Long> sums(List<SourceDocSum> rows) {
Map<String, Long> map = new HashMap<>();
for (SourceDocSum row : rows) {
map.put(row.sourceId(), row.count() == null ? 0L : row.count());
}
return map;
}
private static Map<String, Map<Long, Long>> dailyBySource(List<SourceDayDocSum> rows) {
Map<String, Map<Long, Long>> bySource = new HashMap<>();
for (SourceDayDocSum row : rows) {
bySource.computeIfAbsent(row.sourceId(), key -> new HashMap<>())
.merge(row.day(), row.docs() == null ? 0L : row.docs(), Long::sum);
}
return bySource;
}
}
@@ -74,20 +74,6 @@ public class SourceController {
.orElseGet(() -> ResponseEntity.notFound().build());
}
@GetMapping("/{sourceId}/document-counts")
@Operation(
summary = "Daily document counts for a source",
description =
"The trailing 30-day per-day document series (oldest first) for the source's"
+ " sparkline.")
public ResponseEntity<List<Long>> documentCounts(@PathVariable String sourceId) {
return sourceStore
.get(sourceId)
.filter(sourceAccessGuard::canAccess)
.map(source -> ResponseEntity.ok(overviewService.dailySeries(source.id())))
.orElseGet(() -> ResponseEntity.notFound().build());
}
@PostMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Create or update a source",
@@ -1,7 +0,0 @@
package stirling.software.proprietary.policy.source;
/**
* A {@code (sourceId, epoch-day, summed document count)} row from the daily-aggregate query. The
* day is {@code floor(bucketHour / 24)}, i.e. hours-since-epoch collapsed to days-since-epoch.
*/
public record SourceDayDocSum(String sourceId, Long day, Long docs) {}
@@ -1,62 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import org.springframework.data.domain.Persistable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* One hour's document tally for a source: {@code bucketHour} is the hours-since-epoch the documents
* were fed, {@code docCount} the running total for that hour. Rolling-window totals are summed from
* these buckets. {@code sourceId} is a plain value, not a foreign key, matching the rest of the
* subsystem so it stays decoupled from the security entities.
*/
@Entity
@Table(name = "policy_source_doc_counts")
@IdClass(SourceDocCountId.class)
@NoArgsConstructor
@Getter
@Setter
public class SourceDocCountEntity implements Serializable, Persistable<SourceDocCountId> {
private static final long serialVersionUID = 1L;
@Id
@Column(name = "source_id")
private String sourceId;
@Id
@Column(name = "bucket_hour")
private long bucketHour;
@Column(name = "doc_count")
private long docCount;
public SourceDocCountEntity(String sourceId, long bucketHour, long docCount) {
this.sourceId = sourceId;
this.bucketHour = bucketHour;
this.docCount = docCount;
}
@Override
@Transient
public SourceDocCountId getId() {
return new SourceDocCountId(sourceId, bucketHour);
}
@Override
@Transient
public boolean isNew() {
return true;
}
}
@@ -1,36 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import java.util.Objects;
/** Composite key for {@link SourceDocCountEntity}: one row per source per hour bucket. */
public class SourceDocCountId implements Serializable {
private static final long serialVersionUID = 1L;
private String sourceId;
private long bucketHour;
public SourceDocCountId() {}
public SourceDocCountId(String sourceId, long bucketHour) {
this.sourceId = sourceId;
this.bucketHour = bucketHour;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SourceDocCountId other)) {
return false;
}
return bucketHour == other.bucketHour && Objects.equals(sourceId, other.sourceId);
}
@Override
public int hashCode() {
return Objects.hash(sourceId, bucketHour);
}
}
@@ -1,69 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;
@Repository
public interface SourceDocCountRepository
extends JpaRepository<SourceDocCountEntity, SourceDocCountId> {
/**
* Add to an existing bucket; returns the number of rows updated (0 when the bucket is new).
* Transactional per call so {@code JpaSourceDocCounter.record} can run it (and the retry after
* a concurrent insert) without an enclosing transaction.
*/
@Modifying
@Transactional
@Query(
"update SourceDocCountEntity e set e.docCount = e.docCount + :docs"
+ " where e.sourceId = :sourceId and e.bucketHour = :bucketHour")
int increment(
@Param("sourceId") String sourceId,
@Param("bucketHour") long bucketHour,
@Param("docs") long docs);
/**
* Delete hourly buckets older than {@code floor} (hours-since-epoch). Nothing reads buckets
* before the 30-day window ({@code SourceDocWindows.firstDayHour}); the lifetime total lives in
* {@code policy_source_doc_totals}, so retiring old buckets keeps the table bounded without
* losing any reported figure.
*/
@Modifying
@Transactional
@Query("delete from SourceDocCountEntity e where e.bucketHour < :floor")
int deleteOlderThan(@Param("floor") long floor);
/**
* Document total per source restricted to buckets at or after {@code since} (the 24h window).
*/
@Query(
"select new stirling.software.proprietary.policy.source.SourceDocSum("
+ "e.sourceId, sum(e.docCount))"
+ " from SourceDocCountEntity e"
+ " where e.sourceId in :ids and e.bucketHour >= :since"
+ " group by e.sourceId")
List<SourceDocSum> sumBySourceSince(
@Param("ids") Collection<String> ids, @Param("since") long since);
/**
* Per-source, per-day document totals for buckets at or after {@code since}, summed in the
* database so the overview reads ~one row per source per active day instead of per active hour.
* The day is {@code cast(floor(bucketHour / 24.0) as long)}: {@code 24.0} forces decimal
* division and the cast pins the result to a whole day on every dialect.
*/
@Query(
"select new stirling.software.proprietary.policy.source.SourceDayDocSum("
+ "e.sourceId, cast(floor(e.bucketHour / 24.0) as long), sum(e.docCount))"
+ " from SourceDocCountEntity e"
+ " where e.sourceId in :ids and e.bucketHour >= :since"
+ " group by cast(floor(e.bucketHour / 24.0) as long), e.sourceId")
List<SourceDayDocSum> dailyCountsSince(
@Param("ids") Collection<String> ids, @Param("since") long since);
}
@@ -1,28 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import java.util.Map;
/**
* Records and reports how many documents each source feeds into runs. Counting is bucketed by hour
* so the overview can report rolling totals ({@link DocStats}) cheaply; {@link JpaSourceDocCounter}
* is the runtime bean and {@link InProcessSourceDocCounter} backs tests.
*/
public interface SourceDocCounter {
/** Record that {@code docs} documents were fed from {@code sourceId} at the current time. */
void record(String sourceId, long docs);
/**
* Document totals for each given source; a source with no recorded docs maps to {@link
* DocStats#ZERO}.
*/
Map<String, DocStats> statsFor(Collection<String> sourceIds);
/**
* The trailing {@link DocStats#DAYS}-day daily document series for one source, oldest first,
* for the detail-panel sparkline.
*/
List<Long> dailySeriesFor(String sourceId);
}
@@ -1,6 +0,0 @@
package stirling.software.proprietary.policy.source;
/**
* A {@code (sourceId, summed document count)} row, populated by the doc-count aggregate queries.
*/
public record SourceDocSum(String sourceId, Long count) {}
@@ -1,59 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.io.Serializable;
import org.springframework.data.domain.Persistable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* A source's lifetime document total, denormalized so the overview reads the all-time count in one
* row instead of scanning the source's whole hourly-bucket history, and so {@link
* SourceDocCountEntity} buckets can be pruned to the rolling 30-day window without losing it.
*
* <p>Like {@link SourceDocCountEntity}, implements {@link Persistable} reporting {@code isNew() ==
* true} so a new source's first {@code save} {@code persist}s (a raw INSERT) and a concurrent
* insert surfaces as a constraint violation the counter retries as an increment, rather than {@code
* merge} silently overwriting it.
*/
@Entity
@Table(name = "policy_source_doc_totals")
@NoArgsConstructor
@Getter
@Setter
public class SourceDocTotalEntity implements Serializable, Persistable<String> {
private static final long serialVersionUID = 1L;
@Id
@Column(name = "source_id")
private String sourceId;
@Column(name = "doc_total")
private long docTotal;
public SourceDocTotalEntity(String sourceId, long docTotal) {
this.sourceId = sourceId;
this.docTotal = docTotal;
}
@Override
@Transient
public String getId() {
return sourceId;
}
@Override
@Transient
public boolean isNew() {
return true;
}
}
@@ -1,34 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.util.Collection;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;
@Repository
public interface SourceDocTotalRepository extends JpaRepository<SourceDocTotalEntity, String> {
/**
* Add to a source's lifetime total; returns the number of rows updated (0 when the source has
* no total row yet). Transactional per call so {@code JpaSourceDocCounter.record} can run it
* (and the retry after a concurrent insert) without an enclosing transaction.
*/
@Modifying
@Transactional
@Query(
"update SourceDocTotalEntity e set e.docTotal = e.docTotal + :docs"
+ " where e.sourceId = :sourceId")
int increment(@Param("sourceId") String sourceId, @Param("docs") long docs);
/** Lifetime totals for the given sources, as {@code (sourceId, total)} rows. */
@Query(
"select new stirling.software.proprietary.policy.source.SourceDocSum("
+ "e.sourceId, e.docTotal)"
+ " from SourceDocTotalEntity e where e.sourceId in :ids")
List<SourceDocSum> totalsFor(@Param("ids") Collection<String> ids);
}
@@ -1,61 +0,0 @@
package stirling.software.proprietary.policy.source;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
* Shared rolling-window math over a source's document buckets, so the JPA and in-memory counters
* agree on the window boundaries and the daily series. Both define "last 30 days" as the buckets at
* or after {@link #firstDayHour}, and build the series from per-day counts with {@link #series}
* (the JPA counter aggregates by day in SQL; the in-memory counter groups its hourly buckets with
* {@link #byDay}).
*/
final class SourceDocWindows {
static final long HOURS_IN_24H = 24;
private SourceDocWindows() {}
/**
* The hours-since-epoch at the start of the oldest day in the 30-day window, given the current
* hour bucket. Both the "last 30 days" total and the daily series are measured from here, so
* the KPI and the sparkline always cover the same buckets.
*/
static long firstDayHour(long nowHour) {
return ((nowHour / 24) - (DocStats.DAYS - 1)) * 24;
}
/**
* Build the {@link DocStats#DAYS}-day daily series (oldest first) from per-day document counts
* (keyed by epoch-day, i.e. hours-since-epoch / 24). {@code currentDay} is today's epoch-day;
* the series runs back {@code DAYS} days from it.
*/
static List<Long> series(Map<Long, Long> dailyCounts, long currentDay) {
long firstDay = currentDay - (DocStats.DAYS - 1);
long[] daily = new long[DocStats.DAYS];
for (Map.Entry<Long, Long> day : dailyCounts.entrySet()) {
int dayIndex = (int) (day.getKey() - firstDay);
if (dayIndex >= 0 && dayIndex < DocStats.DAYS) {
daily[dayIndex] += day.getValue();
}
}
return Arrays.stream(daily).boxed().toList();
}
/** Collapse hourly buckets (keyed by hours-since-epoch) into per-day counts (keyed by day). */
static Map<Long, Long> byDay(Map<Long, Long> hourlyCounts) {
Map<Long, Long> dailyCounts = new HashMap<>();
hourlyCounts.forEach((hour, count) -> dailyCounts.merge(hour / 24, count, Long::sum));
return dailyCounts;
}
/** Sum hourly buckets at or after {@code since} (hours-since-epoch). */
static long sumSince(Map<Long, Long> hourlyCounts, long since) {
return hourlyCounts.entrySet().stream()
.filter(entry -> entry.getKey() >= since)
.mapToLong(Map.Entry::getValue)
.sum();
}
}
@@ -30,15 +30,12 @@ public class SourceOverviewService {
private final PolicyStore policyStore;
private final SourceAccessGuard sourceAccessGuard;
private final PolicyAccessGuard policyAccessGuard;
private final SourceDocCounter docCounter;
public SourcesResponse overview() {
List<Source> sources = sourceAccessGuard.visibleFrom(sourceStore);
List<Policy> policies = policyAccessGuard.visibleFrom(policyStore);
Map<String, List<Policy>> referencesBySource = referencesBySource(policies);
Map<String, DocStats> docStats =
docCounter.statsFor(sources.stream().map(Source::id).toList());
List<SourceView> views =
sources.stream()
@@ -47,8 +44,7 @@ public class SourceOverviewService {
toView(
source,
referencesBySource.getOrDefault(
source.id(), List.of()),
docStats.getOrDefault(source.id(), DocStats.ZERO)))
source.id(), List.of())))
.sorted(
Comparator.comparingInt(SourceView::referenceCount)
.reversed()
@@ -58,14 +54,6 @@ public class SourceOverviewService {
return new SourcesResponse(buildKpis(views), views);
}
/**
* The 30-day daily document series for one source (oldest first), for the expanded row's
* sparkline.
*/
public List<Long> dailySeries(String sourceId) {
return docCounter.dailySeriesFor(sourceId);
}
/** Policies referencing each source id, across the caller's visible policies. */
private static Map<String, List<Policy>> referencesBySource(List<Policy> policies) {
Map<String, List<Policy>> bySource = new HashMap<>();
@@ -77,8 +65,7 @@ public class SourceOverviewService {
return bySource;
}
private static SourceView toView(
Source source, List<Policy> referencingPolicies, DocStats docs) {
private static SourceView toView(Source source, List<Policy> referencingPolicies) {
List<SourceView.PolicyRef> refs =
referencingPolicies.stream()
.map(policy -> new SourceView.PolicyRef(policy.id(), policy.name()))
@@ -91,9 +78,7 @@ public class SourceOverviewService {
refs.size(),
refs,
configRows(source),
docs.total(),
docs.last24h(),
docs.last30d());
null);
}
/** A disabled (paused) source reads as "disabled"; an unreferenced one reads as "unused". */
@@ -4,8 +4,8 @@ import java.util.List;
/**
* One row in the Sources overview: a persisted input connection shown exactly once, with how many
* policies reference it (and which) and how many documents it has fed into runs ({@code docsTotal}
* lifetime plus the trailing 24-hour and 30-day windows).
* policies reference it (and which). {@code docsTotal} is {@code null} - per-source document volume
* is not tracked yet; the field is reserved so a later doc-accounting pass is additive.
*/
public record SourceView(
String id,
@@ -15,9 +15,7 @@ public record SourceView(
int referenceCount,
List<PolicyRef> referencingPolicies,
List<DetailRow> config,
long docsTotal,
long docs24h,
long docs30d) {
Long docsTotal) {
/** A policy that references this source. */
public record PolicyRef(String id, String name) {}
@@ -33,11 +33,8 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.storage.repository",
"stirling.software.proprietary.workflow.repository",
"stirling.software.proprietary.policy.store",
"stirling.software.proprietary.policy.source",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.access.repository",
"stirling.software.proprietary.integration.repository",
"stirling.software.proprietary.classification.store"
"stirling.software.proprietary.policy.source"
})
@EntityScan({
"stirling.software.proprietary.security.model",
@@ -45,11 +42,8 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.storage.model",
"stirling.software.proprietary.workflow.model",
"stirling.software.proprietary.policy.store",
"stirling.software.proprietary.policy.source",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.access.model",
"stirling.software.proprietary.integration.model",
"stirling.software.proprietary.classification.store"
"stirling.software.proprietary.policy.source"
})
public class DatabaseConfig {
@@ -28,7 +28,6 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.common.constants.JwtConstants;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.audit.AuditEventType;
import stirling.software.proprietary.audit.AuditLevel;
import stirling.software.proprietary.audit.Audited;
@@ -65,7 +64,6 @@ public class AuthController {
private final ApplicationProperties.Security securityProperties;
private final ApplicationProperties applicationProperties;
private final AiUserDataService aiUserDataService;
private final ResourceAccessService resourceAccessService;
/**
* Login endpoint - replaces Supabase signInWithPassword
@@ -630,7 +628,6 @@ public class AuthController {
userMap.put("username", user.getUsername());
userMap.put("role", user.getRolesAsString());
userMap.put("enabled", user.isEnabled());
userMap.put("portalAccess", resourceAccessService.canAccessPortal(user));
userMap.put(
"authenticationType",
user.getAuthenticationType()); // Expose authentication type for SSO detection
@@ -103,7 +103,6 @@ public class User implements UserDetails, Serializable {
@ElementCollection
@MapKeyColumn(name = "setting_key")
@Lob
@Column(name = "setting_value", columnDefinition = "text")
@CollectionTable(name = "user_settings", joinColumns = @JoinColumn(name = "user_id"))
@JsonIgnore
@@ -352,7 +352,6 @@ public class AiWorkflowService {
try {
List<Resource> resultFiles = new ArrayList<>();
List<Integer> origins = new ArrayList<>();
List<String> inputNames = new ArrayList<>();
for (int i = 0; i < filesToConvert.size(); i++) {
AiFile file = filesToConvert.get(i);
@@ -377,15 +376,11 @@ public class AiWorkflowService {
definition,
PolicyInputs.of(List.of(input)),
PolicyProgressListener.NOOP);
// Each conversion runs on one input, so every output traces back to file i.
for (Resource output : result.files()) {
resultFiles.add(output);
origins.add(i);
}
resultFiles.addAll(result.files());
inputNames.add(multipartFile.getOriginalFilename());
}
return new WorkflowState.Terminal(
buildCompletedResponse(null, resultFiles, origins, inputNames, null));
buildCompletedResponse(null, resultFiles, inputNames, null));
} catch (InternalApiTimeoutException e) {
log.error("PDF to Markdown conversion timed out: {}", e.getMessage());
return new WorkflowState.Terminal(
@@ -477,7 +472,6 @@ public class AiWorkflowService {
buildCompletedResponse(
response.getRationale(),
result.files(),
result.origins(),
inputFileNames(filesById),
result.report()));
} catch (InternalApiTimeoutException e) {
@@ -539,8 +533,7 @@ public class AiWorkflowService {
}
};
return new WorkflowState.Terminal(
buildCompletedResponse(
response.getSummary(), List.of(resource), null, List.of(), null));
buildCompletedResponse(response.getSummary(), List.of(resource), List.of(), null));
}
@SuppressWarnings("unchecked")
@@ -598,11 +591,7 @@ public class AiWorkflowService {
return new WorkflowState.Terminal(
buildCompletedResponse(
summary,
result.files(),
result.origins(),
inputFileNames(filesById),
result.report()));
summary, result.files(), inputFileNames(filesById), result.report()));
} catch (InternalApiTimeoutException e) {
log.error("Plan step on tool {} timed out: {}", e.getEndpointPath(), e.getMessage());
return new WorkflowState.Terminal(
@@ -691,35 +680,19 @@ public class AiWorkflowService {
private AiWorkflowResponse buildCompletedResponse(
String summary,
List<Resource> resultFiles,
List<Integer> origins,
List<String> inputFileNames,
JsonNode report)
throws IOException {
// Store every output file individually so each gets its own Stirling file ID and the
// frontend can add them as independent variants without going through a zip.
// Count outputs per source so only a clean 1:1 transform (one output for a source) reuses
// the input's name; a split (one input → many outputs) keeps each entry's own name.
Map<Integer, Long> outputsPerOrigin =
origins == null
? Map.of()
: origins.stream()
.filter(o -> o != null)
.collect(Collectors.groupingBy(o -> o, Collectors.counting()));
boolean preserveInputNames = inputFileNames.size() == resultFiles.size();
List<AiWorkflowResultFile> descriptors = new ArrayList<>();
for (int i = 0; i < resultFiles.size(); i++) {
Resource resource = resultFiles.get(i);
String responseName = resource.getFilename();
// The output's source input (from the executor), used both to name it and to tell the
// client which file to version in place.
Integer origin = origins != null && i < origins.size() ? origins.get(i) : null;
boolean uniqueOrigin =
origin != null && outputsPerOrigin.getOrDefault(origin, 0L) == 1L;
String inputName =
uniqueOrigin && origin >= 0 && origin < inputFileNames.size()
? inputFileNames.get(origin)
: null;
// Prefer the source input's name only for 1:1 operations where the output keeps the
// same extension (rotate, compress, etc.). For converters and other extension-changing
String inputName = preserveInputNames ? inputFileNames.get(i) : null;
// Prefer the input name only for 1:1 operations where the output keeps the same
// extension (rotate, compress, etc.). For converters and other extension-changing
// tools, the response filename from Content-Disposition is authoritative.
String name;
if (inputName != null
@@ -739,11 +712,7 @@ public class AiWorkflowService {
try (java.io.InputStream is = resource.getInputStream()) {
fileId = fileStorage.storeInputStream(is, name).fileId();
}
// Only expose the source when this is a clean 1:1 transform, so the client can treat a
// present sourceIndex as "replace that input in place" without further disambiguation.
descriptors.add(
new AiWorkflowResultFile(
fileId, name, contentType, uniqueOrigin ? origin : null));
descriptors.add(new AiWorkflowResultFile(fileId, name, contentType));
}
AiWorkflowResponse completed = new AiWorkflowResponse();
@@ -1,174 +0,0 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import java.util.Optional;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.proprietary.access.model.OwnedResource;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.TeamRepository;
@ExtendWith(MockitoExtension.class)
class OwnershipServiceTest {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
@Mock private ResourceAccessService accessService;
@Mock private TeamLeadLookup teamLeadLookup;
@Mock private TeamRepository teamRepository;
@InjectMocks private OwnershipService ownership;
/** Minimal concrete OwnedResource for exercising the base behaviour. */
static class TestResource extends OwnedResource {
private final Long id;
TestResource(Long id) {
this.id = id;
}
@Override
public Long getId() {
return id;
}
}
// ---- scope-based create authorization ----
@Test
void userScopeSetsOwner() {
User user = user(7);
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.USER, null, user, () -> false);
assertThat(r.getScope()).isEqualTo(OwnerScope.USER);
assertThat(r.getOwnerUser()).isSameAs(user);
}
@Test
void userScopeBlockedByLockedServerOverride() {
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L), OwnerScope.USER, null, user(7), () -> true));
}
@Test
void adminMayCreateServerScope() {
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.SERVER, null, admin(1), () -> false);
assertThat(r.getOwnerUser()).isNull();
}
@Test
void nonAdminCannotCreateServerScope() {
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L),
OwnerScope.SERVER,
null,
user(7),
() -> false));
}
@Test
void teamLeaderMayCreateTeamScope() {
Team team = new Team();
team.setId(5L);
User user = user(7);
when(teamRepository.findById(5L)).thenReturn(Optional.of(team));
when(teamLeadLookup.isLeaderOfTeam(user, 5L)).thenReturn(true);
TestResource r = new TestResource(1L);
ownership.assignOwnership(r, OwnerScope.TEAM, 5L, user, () -> false);
assertThat(r.getOwnerTeam()).isSameAs(team);
}
@Test
void nonLeaderCannotCreateTeamScope() {
Team team = new Team();
team.setId(5L);
when(teamRepository.findById(5L)).thenReturn(Optional.of(team));
assertForbidden(
() ->
ownership.assignOwnership(
new TestResource(1L), OwnerScope.TEAM, 5L, user(7), () -> false));
}
// ---- use / manage ----
@Test
void enabledResourceUseDelegatesToTheAcl() {
TestResource r = new TestResource(1L);
User user = user(7);
when(accessService.canUseResource(any(), any(), any(), any(), any())).thenReturn(true);
assertThat(ownership.canUse(TYPE, r, user)).isTrue();
}
@Test
void disabledResourceUsableOnlyByOwnerOrAdmin() {
TestResource r = new TestResource(1L);
r.setEnabled(false);
r.setOwnerUser(user(7));
assertThat(ownership.canUse(TYPE, r, user(7))).isTrue(); // owner
assertThat(ownership.canUse(TYPE, r, user(8))).isFalse(); // someone else
assertThat(ownership.canUse(TYPE, r, admin(2))).isTrue(); // admin
}
@Test
void disabledTeamResourceUsableByLeaderOfOwningTeam() {
Team team = new Team();
team.setId(5L);
TestResource r = new TestResource(1L);
r.setEnabled(false);
r.setOwnerTeam(team);
User leader = user(7);
when(teamLeadLookup.isLeaderOfTeam(leader, 5L)).thenReturn(true);
assertThat(ownership.canUse(TYPE, r, leader)).isTrue(); // lead of the owning team
assertThat(ownership.canUse(TYPE, r, user(8))).isFalse(); // not a lead
}
// ---- helpers ----
private void assertForbidden(org.assertj.core.api.ThrowableAssert.ThrowingCallable call) {
assertThatThrownBy(call)
.isInstanceOf(ResponseStatusException.class)
.extracting(e -> ((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.FORBIDDEN);
}
private User user(long id) {
User u = new User();
u.setId(id);
u.setUsername("user" + id);
return u;
}
private User admin(long id) {
User u = user(id);
new Authority("ROLE_ADMIN", u);
return u;
}
}
@@ -1,217 +0,0 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.when;
import java.lang.reflect.Field;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.proprietary.access.model.AccessPermission;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.PrincipalType;
import stirling.software.proprietary.access.model.ResourceGrant;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.repository.ResourceGrantRepository;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
@ExtendWith(MockitoExtension.class)
class ResourceAccessServiceTest {
private static final ResourceType TYPE = ResourceType.INTEGRATION_CONFIG;
private static final String RID = "42";
@Mock private ResourceGrantRepository grantRepository;
@Mock private TeamLeadLookup teamLeadLookup;
@InjectMocks private ResourceAccessService service;
@BeforeEach
void setPortalDefault() throws Exception {
Field f = ResourceAccessService.class.getDeclaredField("portalDefaultPolicy");
f.setAccessible(true);
f.set(service, DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS);
}
// ---- owner / admin short-circuits ----
@Test
void adminMayUseEvenWithExplicitOnlyAndNoGrants() {
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, admin(1)))
.isTrue();
}
@Test
void ownerMayUseEvenWithExplicitOnly() {
assertThat(
service.canUseResource(
TYPE, RID, 5L, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void nullUserIsAlwaysDenied() {
assertThat(service.canUseResource(TYPE, RID, 5L, DefaultAccessPolicy.ORG_ALL, null))
.isFalse();
assertThat(service.canManageResource(TYPE, RID, 5L, null)).isFalse();
}
// ---- explicit grants ----
@Test
void explicitUserGrantAllowsUse() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void teamGrantAllowsUseForTeamMember() {
stubGrants(grant(PrincipalType.TEAM, 7L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.EXPLICIT_ONLY,
userInTeam(5, 7)))
.isTrue();
}
@Test
void teamGrantDoesNotLeakToOtherTeams() {
stubGrants(grant(PrincipalType.TEAM, 7L, AccessPermission.USE));
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.EXPLICIT_ONLY,
userInTeam(5, 99)))
.isFalse();
}
@Test
void manageGrantImpliesUse() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.MANAGE));
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isTrue();
}
@Test
void useGrantDoesNotImplyManage() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.USE));
assertThat(service.canManageResource(TYPE, RID, null, user(5))).isFalse();
}
@Test
void manageGrantAllowsManage() {
stubGrants(grant(PrincipalType.USER, 5L, AccessPermission.MANAGE));
assertThat(service.canManageResource(TYPE, RID, null, user(5))).isTrue();
}
// ---- default policies ----
@Test
void orgAllDefaultAllowsAnyUser() {
stubGrants();
assertThat(service.canUseResource(TYPE, RID, null, DefaultAccessPolicy.ORG_ALL, user(5)))
.isTrue();
}
@Test
void explicitOnlyDefaultDeniesUngrantedUser() {
stubGrants();
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.EXPLICIT_ONLY, user(5)))
.isFalse();
}
@Test
void teamLeadDefaultAllowsLeaderButNotRegularUser() {
stubGrants();
User leader = user(5);
when(teamLeadLookup.isAnyTeamLeader(leader)).thenReturn(true);
assertThat(
service.canUseResource(
TYPE, RID, null, DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS, leader))
.isTrue();
stubGrants();
assertThat(
service.canUseResource(
TYPE,
RID,
null,
DefaultAccessPolicy.ADMINS_AND_TEAM_LEADS,
user(6)))
.isFalse();
}
// ---- portal convenience (default policy ADMINS_AND_TEAM_LEADS) ----
@Test
void portalAccessibleByAdmin() {
assertThat(service.canAccessPortal(admin(1))).isTrue();
}
@Test
void portalDeniedToRegularUser() {
when(grantRepository.findByResourceTypeAndResourceId(ResourceType.PORTAL, ""))
.thenReturn(List.of());
assertThat(service.canAccessPortal(user(5))).isFalse();
}
// ---- helpers ----
private void stubGrants(ResourceGrant... grants) {
when(grantRepository.findByResourceTypeAndResourceId(TYPE, RID))
.thenReturn(List.of(grants));
}
private ResourceGrant grant(PrincipalType type, long principalId, AccessPermission permission) {
ResourceGrant g = new ResourceGrant();
g.setResourceType(TYPE);
g.setResourceId(RID);
g.setPrincipalType(type);
g.setPrincipalId(principalId);
g.setPermission(permission);
return g;
}
private User user(long id) {
User u = new User();
u.setId(id);
u.setUsername("user" + id);
return u;
}
private User admin(long id) {
User u = user(id);
new Authority("ROLE_ADMIN", u);
return u;
}
private User userInTeam(long id, long teamId) {
User u = user(id);
Team team = new Team();
team.setId(teamId);
u.setTeam(team);
return u;
}
}
@@ -1,80 +0,0 @@
package stirling.software.proprietary.access.service;
import static org.assertj.core.api.Assertions.assertThat;
import java.util.LinkedHashMap;
import java.util.Map;
import org.junit.jupiter.api.Test;
class SecretMaskerTest {
private final SecretMasker masker = new SecretMasker();
@Test
void maskHidesFlatAndNestedSecretsButKeepsPlainFields() {
Map<String, Object> nested = new LinkedHashMap<>();
nested.put("host", "h.example");
nested.put("secretKey", "sk-real");
Map<String, Object> config = new LinkedHashMap<>();
config.put("bucket", "acme");
config.put("accessKey", "AKIA123");
config.put("connection", nested); // non-sensitive parent, sensitive child
Map<String, Object> masked = masker.mask(config);
assertThat(masked.get("bucket")).isEqualTo("acme");
assertThat(masked.get("accessKey")).isEqualTo(SecretMasker.MASK);
@SuppressWarnings("unchecked")
Map<String, Object> conn = (Map<String, Object>) masked.get("connection");
assertThat(conn.get("host")).isEqualTo("h.example");
assertThat(conn.get("secretKey")).isEqualTo(SecretMasker.MASK);
}
@Test
void mergeKeepsStoredSecretWhenIncomingIsMasked() {
Map<String, Object> stored = Map.of("bucket", "old", "secretKey", "REAL");
Map<String, Object> incoming = Map.of("bucket", "new", "secretKey", SecretMasker.MASK);
Map<String, Object> merged = masker.merge(stored, incoming);
assertThat(merged.get("bucket")).isEqualTo("new"); // non-secret updated
assertThat(merged.get("secretKey")).isEqualTo("REAL"); // secret retained
}
@Test
void mergeAcceptsARealNewSecret() {
Map<String, Object> merged =
masker.merge(Map.of("secretKey", "OLD"), Map.of("secretKey", "NEW"));
assertThat(merged.get("secretKey")).isEqualTo("NEW");
}
@Test
void sanitizeDropsBlankSecretsOnCreate() {
Map<String, Object> incoming = new LinkedHashMap<>();
incoming.put("bucket", "b");
incoming.put("secretKey", "");
Map<String, Object> clean = masker.sanitize(incoming);
assertThat(clean).containsEntry("bucket", "b").doesNotContainKey("secretKey");
}
@Test
void deeplyNestedInputIsBoundedNotOverflowing() {
// Build a structure far deeper than the recursion cap.
Map<String, Object> root = new LinkedHashMap<>();
Map<String, Object> cur = root;
for (int i = 0; i < 2000; i++) {
Map<String, Object> next = new LinkedHashMap<>();
cur.put("child", next);
cur = next;
}
cur.put("secretKey", "deep");
// Must return (bounded recursion), not throw StackOverflowError.
assertThat(masker.mask(root)).isNotNull();
assertThat(masker.sanitize(root)).isNotNull();
assertThat(masker.merge(root, root)).isNotNull();
}
}
@@ -1,134 +0,0 @@
package stirling.software.proprietary.classification;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.Mockito.when;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.classification.model.ClassificationTaxonomy;
import stirling.software.proprietary.classification.model.TaxonomyCategory;
import stirling.software.proprietary.classification.model.TaxonomyDocumentType;
import stirling.software.proprietary.classification.store.InProcessTaxonomyStore;
import stirling.software.proprietary.classification.store.TaxonomyStore;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
@ExtendWith(MockitoExtension.class)
@DisplayName("TaxonomyController")
class TaxonomyControllerTest {
private static final Long TEAM = 7L;
@Mock private PolicyManagementAuthority policyManagementAuthority;
@Mock private UserServiceInterface userService;
private TaxonomyStore store;
private ApplicationProperties applicationProperties;
private TaxonomyController controller;
@BeforeEach
void setUp() {
store = new InProcessTaxonomyStore();
applicationProperties = new ApplicationProperties();
controller =
new TaxonomyController(
store, policyManagementAuthority, applicationProperties, userService);
}
private static ClassificationTaxonomy sample() {
return new ClassificationTaxonomy(
List.of(
new TaxonomyCategory(
"invoice",
"Invoice",
List.of(new TaxonomyDocumentType("receipt", "Receipt")))),
List.of("finance"));
}
private void loginEnabled(boolean enabled) {
applicationProperties.getSecurity().setEnableLogin(enabled);
}
@Test
@DisplayName("GET returns 204 when the team has no taxonomy")
void getEmpty() {
when(policyManagementAuthority.currentUserTeamId()).thenReturn(TEAM);
ResponseEntity<ClassificationTaxonomy> response = controller.getTaxonomy();
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.NO_CONTENT);
}
@Test
@DisplayName("PUT then GET round-trips the team's taxonomy (login disabled)")
void saveThenGet() {
loginEnabled(false);
when(policyManagementAuthority.currentUserTeamId()).thenReturn(TEAM);
controller.saveTaxonomy(sample());
ResponseEntity<ClassificationTaxonomy> got = controller.getTaxonomy();
assertThat(got.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(got.getBody()).isNotNull();
assertThat(got.getBody().categories()).hasSize(1);
assertThat(got.getBody().categories().getFirst().id()).isEqualTo("invoice");
}
@Test
@DisplayName("PUT is scoped per team")
void perTeam() {
loginEnabled(false);
when(policyManagementAuthority.currentUserTeamId()).thenReturn(TEAM);
controller.saveTaxonomy(sample());
when(policyManagementAuthority.currentUserTeamId()).thenReturn(99L);
assertThat(controller.getTaxonomy().getStatusCode()).isEqualTo(HttpStatus.NO_CONTENT);
}
@Test
@DisplayName("PUT is rejected for a non-editor when login is enabled")
void putForbiddenForNonEditor() {
loginEnabled(true);
when(policyManagementAuthority.canEditPolicies()).thenReturn(false);
assertThatThrownBy(() -> controller.saveTaxonomy(sample()))
.isInstanceOf(ResponseStatusException.class)
.hasFieldOrPropertyWithValue("statusCode", HttpStatus.FORBIDDEN);
}
@Test
@DisplayName("PUT rejects an invalid taxonomy with 400")
void putInvalid() {
loginEnabled(false);
assertThatThrownBy(
() ->
controller.saveTaxonomy(
new ClassificationTaxonomy(List.of(), List.of())))
.isInstanceOf(ResponseStatusException.class)
.hasFieldOrPropertyWithValue("statusCode", HttpStatus.BAD_REQUEST);
}
@Test
@DisplayName("DELETE resets the team back to no stored taxonomy")
void deleteResets() {
loginEnabled(false);
when(policyManagementAuthority.currentUserTeamId()).thenReturn(TEAM);
controller.saveTaxonomy(sample());
ResponseEntity<Void> response = controller.resetTaxonomy();
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.NO_CONTENT);
assertThat(controller.getTaxonomy().getStatusCode()).isEqualTo(HttpStatus.NO_CONTENT);
}
}
@@ -1,110 +0,0 @@
package stirling.software.proprietary.classification.model;
import static org.assertj.core.api.Assertions.assertThatCode;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@DisplayName("TaxonomyValidator")
class TaxonomyValidatorTest {
private static TaxonomyCategory category(String id, TaxonomyDocumentType... docTypes) {
return new TaxonomyCategory(id, id + " label", List.of(docTypes));
}
private static TaxonomyDocumentType docType(String id) {
return new TaxonomyDocumentType(id, id + " label");
}
@Test
@DisplayName("accepts a well-formed taxonomy")
void acceptsValid() {
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(category("invoice", docType("receipt")), category("contract")),
List.of("finance", "legal"));
assertThatCode(() -> TaxonomyValidator.validate(taxonomy)).doesNotThrowAnyException();
}
@Test
@DisplayName("rejects a taxonomy with no categories")
void rejectsEmpty() {
ClassificationTaxonomy taxonomy = new ClassificationTaxonomy(List.of(), List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("at least one category");
}
@Test
@DisplayName("rejects duplicate category ids")
void rejectsDuplicateCategory() {
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(category("invoice"), category("invoice")), List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("Duplicate category id");
}
@Test
@DisplayName("rejects duplicate doc type ids within a category")
void rejectsDuplicateDocType() {
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(category("invoice", docType("receipt"), docType("receipt"))),
List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("Duplicate doc type id");
}
@Test
@DisplayName("rejects blank ids and labels")
void rejectsBlank() {
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(new TaxonomyCategory(" ", "label", List.of())), List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("must not be blank");
}
@Test
@DisplayName("rejects duplicate tags")
void rejectsDuplicateTags() {
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(category("invoice")), List.of("finance", "finance"));
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("Duplicate tag");
}
@Test
@DisplayName("rejects more categories than the cap")
void rejectsTooManyCategories() {
List<TaxonomyCategory> categories =
java.util.stream.IntStream.rangeClosed(0, TaxonomyValidator.MAX_CATEGORIES)
.mapToObj(i -> category("cat" + i))
.toList();
ClassificationTaxonomy taxonomy = new ClassificationTaxonomy(categories, List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("Too many categories");
}
@Test
@DisplayName("rejects an over-long label")
void rejectsOverLongLabel() {
String longLabel = "x".repeat(TaxonomyValidator.MAX_TEXT_LENGTH + 1);
ClassificationTaxonomy taxonomy =
new ClassificationTaxonomy(
List.of(new TaxonomyCategory("invoice", longLabel, List.of())), List.of());
assertThatThrownBy(() -> TaxonomyValidator.validate(taxonomy))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("too long");
}
}
@@ -1,105 +0,0 @@
package stirling.software.proprietary.controller.api;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.util.TempFileManager;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.PdfContentExtractor;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class ClassifyTagControllerTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
@Mock private TempFileManager tempFileManager;
@Mock private PdfContentExtractor pdfContentExtractor;
@Mock private PdfMetadataService pdfMetadataService;
@Mock private AiEngineClient aiEngineClient;
private final ObjectMapper objectMapper = JsonMapper.builder().build();
private ClassifyTagController controller;
@BeforeEach
void setUp() {
controller =
new ClassifyTagController(
pdfDocumentFactory,
tempFileManager,
pdfContentExtractor,
pdfMetadataService,
aiEngineClient,
objectMapper,
null,
null,
null);
}
@Test
void classifyAndTag_writesClassificationWithoutOutcome() throws Exception {
PDDocument document = mock(PDDocument.class);
MultipartFile file = mock(MultipartFile.class);
when(file.getOriginalFilename()).thenReturn("invoice.pdf");
when(pdfDocumentFactory.load(any(MultipartFile.class), eq(true))).thenReturn(document);
when(document.getNumberOfPages()).thenReturn(1);
when(pdfContentExtractor.extractPageTextRaw(document, 1))
.thenReturn("Invoice total due 100.00");
when(aiEngineClient.post(eq("/api/v1/documents/classify"), anyString(), isNull()))
.thenReturn(
"{\"outcome\":\"classification\",\"category\":\"invoice\","
+ "\"docType\":\"invoice\",\"typeConfidence\":0.98,"
+ "\"tags\":[\"finance\"]}");
try {
controller.classifyAndTag(file);
} catch (Exception ignored) {
// WebResponseUtils.pdfDocToWebResponse needs a real temp file; the metadata write we
// assert on has already happened by the time it runs.
}
ArgumentCaptor<String> value = ArgumentCaptor.forClass(String.class);
verify(pdfMetadataService).setClassificationMetadata(eq(document), value.capture());
JsonNode written = objectMapper.readTree(value.getValue());
assertThat(written.has("outcome")).isFalse();
assertThat(written.get("category").asText()).isEqualTo("invoice");
assertThat(written.get("docType").asText()).isEqualTo("invoice");
assertThat(written.get("tags").get(0).asText()).isEqualTo("finance");
}
@Test
void windowPageNumbers_takesFirstAndLastWithoutOverlap() {
assertEquals(List.of(1, 2, 4, 5), ClassifyTagController.windowPageNumbers(5, 2));
assertEquals(List.of(1, 2, 3), ClassifyTagController.windowPageNumbers(3, 2));
// Short docs clamp + dedupe rather than throwing or going out of range.
assertEquals(List.of(1, 2), ClassifyTagController.windowPageNumbers(2, 2));
assertEquals(List.of(1), ClassifyTagController.windowPageNumbers(1, 2));
assertEquals(List.of(), ClassifyTagController.windowPageNumbers(0, 2));
}
}
@@ -1,64 +0,0 @@
package stirling.software.proprietary.integration.crypto;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import java.util.Base64;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
class CredentialEncryptionTest {
@BeforeAll
static void initKey() throws Exception {
KeyGenerator generator = KeyGenerator.getInstance("AES");
generator.init(256);
SecretKey key = generator.generateKey();
CredentialEncryption.initialiseForTesting(key);
}
@Test
void roundTripRecoversPlaintext() {
String plaintext = "s3-secret-key-ABC123/+=value";
String encrypted = CredentialEncryption.encrypt(plaintext);
assertThat(encrypted).isNotNull().isNotEqualTo(plaintext);
assertThat(CredentialEncryption.decrypt(encrypted)).isEqualTo(plaintext);
}
@Test
void sameInputProducesDifferentCiphertext() {
String plaintext = "repeated-secret";
// Random IV per encryption => ciphertext must differ, but both decrypt back.
String first = CredentialEncryption.encrypt(plaintext);
String second = CredentialEncryption.encrypt(plaintext);
assertThat(first).isNotEqualTo(second);
assertThat(CredentialEncryption.decrypt(first)).isEqualTo(plaintext);
assertThat(CredentialEncryption.decrypt(second)).isEqualTo(plaintext);
}
@Test
void nullsPassThrough() {
assertThat(CredentialEncryption.encrypt(null)).isNull();
assertThat(CredentialEncryption.decrypt(null)).isNull();
}
@Test
void tamperedCiphertextIsRejected() {
// GCM is authenticated: flipping a byte of the stored blob must fail decryption, not
// silently return corrupted plaintext.
String encrypted = CredentialEncryption.encrypt("top-secret");
byte[] raw = Base64.getDecoder().decode(encrypted);
raw[raw.length - 1] ^= 0x01;
String tampered = Base64.getEncoder().encodeToString(raw);
assertThatThrownBy(() -> CredentialEncryption.decrypt(tampered))
.isInstanceOf(IllegalStateException.class);
}
}
@@ -1,182 +0,0 @@
package stirling.software.proprietary.integration.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.access.service.SecretMasker;
import stirling.software.proprietary.integration.dto.IntegrationConfigRequest;
import stirling.software.proprietary.integration.dto.IntegrationConfigResponse;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.ObjectMapper;
/**
* Integration-glue tests. Ownership authorization and secret masking are covered by {@code
* OwnershipServiceTest} / {@code SecretMaskerTest}; here those collaborators are mocked and we
* assert this service delegates to them correctly.
*/
@ExtendWith(MockitoExtension.class)
class IntegrationConfigServiceTest {
private static final ObjectMapper MAPPER = new ObjectMapper();
@Mock private IntegrationConfigRepository repository;
@Mock private OwnershipService ownership;
@Mock private SecretMasker secretMasker;
@InjectMocks private IntegrationConfigService service;
@Test
void createDelegatesOwnershipAndSanitizesConfig() {
when(secretMasker.sanitize(any())).thenReturn(Map.of("bucket", "b"));
when(repository.save(any())).thenAnswer(inv -> inv.getArgument(0));
User user = user(7);
IntegrationConfig created =
service.create(request(IntegrationType.S3, OwnerScope.USER, null), user);
assertThat(created.getIntegrationType()).isEqualTo(IntegrationType.S3);
assertThat(created.getName()).isEqualTo("name");
verify(ownership)
.assignOwnership(eq(created), eq(OwnerScope.USER), isNull(), eq(user), any());
verify(secretMasker).sanitize(any());
}
@Test
void updateMergesConfigViaSecretMasker() throws Exception {
IntegrationConfig cfg = config(2L);
cfg.setConfig("{\"bucket\":\"old\",\"secretKey\":\"REAL\"}");
when(repository.findById(2L)).thenReturn(Optional.of(cfg));
when(ownership.canManage(any(), eq(cfg), any())).thenReturn(true);
when(secretMasker.merge(any(), any()))
.thenReturn(Map.of("bucket", "new", "secretKey", "REAL"));
when(repository.save(any())).thenAnswer(inv -> inv.getArgument(0));
IntegrationConfigRequest req =
new IntegrationConfigRequest(
null,
null,
null,
null,
null,
null,
null,
Map.of("bucket", "new", "secretKey", "********"));
service.update(2L, req, user(7));
Map<String, Object> stored =
MAPPER.readValue(cfg.getConfig(), new TypeReference<Map<String, Object>>() {});
assertThat(stored.get("bucket")).isEqualTo("new");
assertThat(stored.get("secretKey")).isEqualTo("REAL");
}
@Test
void updateForbiddenWhenCannotManage() {
IntegrationConfig cfg = config(3L);
when(repository.findById(3L)).thenReturn(Optional.of(cfg));
when(ownership.canManage(any(), eq(cfg), any())).thenReturn(false);
assertThatThrownBy(
() ->
service.update(
3L,
request(IntegrationType.S3, OwnerScope.USER, null),
user(7)))
.isInstanceOf(ResponseStatusException.class)
.extracting(e -> ((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.FORBIDDEN);
}
@Test
void toResponseMasksConfig() {
IntegrationConfig cfg = config(4L);
cfg.setConfig("{\"secretKey\":\"x\"}");
when(secretMasker.mask(any())).thenReturn(Map.of("secretKey", SecretMasker.MASK));
when(ownership.canManage(any(), eq(cfg), any())).thenReturn(false);
IntegrationConfigResponse resp = service.toResponse(cfg, user(7));
assertThat(resp.config().get("secretKey")).isEqualTo(SecretMasker.MASK);
assertThat(resp.canManage()).isFalse();
}
@Test
void listIncludesAConfigSharedViaAnExplicitGrant() {
User user = user(7);
when(repository.findByOwnerUser(user)).thenReturn(List.of());
when(repository.findByScope(OwnerScope.SERVER)).thenReturn(List.of());
when(ownership.grantedResourceIds(any(), eq(user))).thenReturn(Set.of("99"));
IntegrationConfig shared = config(99L);
when(repository.findById(99L)).thenReturn(Optional.of(shared));
when(ownership.canUse(any(), eq(shared), eq(user))).thenReturn(true);
List<IntegrationConfig> visible = service.listVisible(user);
assertThat(visible).extracting(IntegrationConfig::getId).contains(99L);
}
@Test
void nonAdminCannotChangeLockedFlag() {
IntegrationConfig cfg = config(5L); // locked = false
when(repository.findById(5L)).thenReturn(Optional.of(cfg));
when(ownership.canManage(any(), eq(cfg), any())).thenReturn(true);
when(ownership.isAdmin(any())).thenReturn(false);
IntegrationConfigRequest req =
new IntegrationConfigRequest(null, null, null, null, null, true, null, null);
assertThatThrownBy(() -> service.update(5L, req, user(7)))
.isInstanceOf(ResponseStatusException.class)
.extracting(e -> ((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.FORBIDDEN);
}
// ---- helpers ----
private IntegrationConfig config(long id) {
IntegrationConfig cfg = new IntegrationConfig();
cfg.setId(id);
cfg.setIntegrationType(IntegrationType.S3);
cfg.setName("cfg" + id);
cfg.setEnabled(true);
cfg.setDefaultAccess(DefaultAccessPolicy.EXPLICIT_ONLY);
return cfg;
}
private IntegrationConfigRequest request(IntegrationType type, OwnerScope scope, Long teamId) {
return new IntegrationConfigRequest(
type, "name", scope, teamId, null, null, null, Map.of("bucket", "b"));
}
private User user(long id) {
User u = new User();
u.setId(id);
u.setUsername("user" + id);
return u;
}
}
@@ -33,7 +33,6 @@ import stirling.software.proprietary.policy.model.PolicyInputs;
import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.model.PolicyRunStatus;
import stirling.software.proprietary.policy.progress.PolicyProgressListener;
import stirling.software.proprietary.policy.source.InProcessSourceDocCounter;
import stirling.software.proprietary.policy.source.InProcessSourceStore;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceStore;
@@ -54,12 +53,7 @@ class PolicyRunnerTest {
@BeforeEach
void setUp() {
runner =
new PolicyRunner(
policyEngine,
List.of(folderSource),
sourceStore,
new InProcessSourceDocCounter());
runner = new PolicyRunner(policyEngine, List.of(folderSource), sourceStore);
}
@Test
@@ -1,62 +0,0 @@
package stirling.software.proprietary.policy.source;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.time.Instant;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.jupiter.api.Test;
/**
* Tests the rolling-window aggregation: documents recorded at different times roll into the total,
* the 24-hour / 30-day windows, and the daily series correctly as the clock advances.
*/
class InProcessSourceDocCounterTest {
private static InProcessSourceDocCounter seededCounter(AtomicReference<Instant> clock) {
InProcessSourceDocCounter counter = new InProcessSourceDocCounter(clock::get);
// 40 days ago: outside the 30-day window.
clock.set(Instant.parse("2026-05-21T12:00:00Z"));
counter.record("s", 100);
// 10 days ago: inside 30 days, outside 24 hours.
clock.set(Instant.parse("2026-06-20T12:00:00Z"));
counter.record("s", 50);
// 2 hours ago: inside both windows.
clock.set(Instant.parse("2026-06-30T10:00:00Z"));
counter.record("s", 7);
// Query as of "now".
clock.set(Instant.parse("2026-06-30T12:00:00Z"));
return counter;
}
@Test
void rollsCountsIntoTotalAnd24hAnd30dWindows() {
AtomicReference<Instant> clock = new AtomicReference<>();
DocStats stats = seededCounter(clock).statsFor(List.of("s")).get("s");
assertEquals(157, stats.total());
assertEquals(7, stats.last24h());
assertEquals(57, stats.last30d());
}
@Test
void buildsTheDailySeriesOldestFirst() {
AtomicReference<Instant> clock = new AtomicReference<>();
// Today (index 29) and 10 days ago (index 19) only; 40 days ago is outside the window.
List<Long> series = seededCounter(clock).dailySeriesFor("s");
assertEquals(30, series.size());
assertEquals(7L, series.get(29));
assertEquals(50L, series.get(19));
assertEquals(57L, series.stream().mapToLong(Long::longValue).sum());
}
@Test
void aSourceWithNoRecordedDocsIsZero() {
InProcessSourceDocCounter counter = new InProcessSourceDocCounter();
assertEquals(DocStats.ZERO, counter.statsFor(List.of("unknown")).get("unknown"));
assertEquals(Collections.nCopies(DocStats.DAYS, 0L), counter.dailySeriesFor("unknown"));
}
}
@@ -1,93 +0,0 @@
package stirling.software.proprietary.policy.source;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.time.Instant;
import java.util.List;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.autoconfigure.AutoConfigurationPackage;
import org.springframework.boot.data.jpa.test.autoconfigure.DataJpaTest;
/**
* Exercises {@link JpaSourceDocCounter} against a real (H2) database so the daily-aggregate query
* ({@code cast(floor(bucketHour / 24.0) as long)} grouping) and the record upsert are actually run,
* not just asserted against a mock. The unit-level window math lives in {@link
* InProcessSourceDocCounterTest}.
*/
@DataJpaTest
class JpaSourceDocCounterDbTest {
// Pinned "now" so seeding and statsFor agree regardless of when the test runs (no hour-tick
// flake at the boundary of a wall-clock hour).
private static final Instant NOW = Instant.parse("2026-06-30T12:00:00Z");
private static final long NOW_HOUR = NOW.getEpochSecond() / 3600;
@Autowired private SourceDocCountRepository repository;
@Autowired private SourceDocTotalRepository totalRepository;
private JpaSourceDocCounter counter() {
return new JpaSourceDocCounter(repository, totalRepository, () -> NOW);
}
@Test
void recordIncrementsBothTheHourlyBucketAndTheLifetimeTotal() {
JpaSourceDocCounter counter = counter();
counter.record("s", 5);
counter.record("s", 3);
DocStats stats = counter.statsFor(List.of("s")).get("s");
assertEquals(8, stats.total()); // from the denormalized lifetime row
assertEquals(8, stats.last24h());
assertEquals(8, stats.last30d());
assertEquals(8L, counter.dailySeriesFor("s").get(DocStats.DAYS - 1));
}
@Test
void statsBucketDocsByDayAndWindowFromTheDatabase() {
// Seed buckets directly at controlled hours: today, 10 days ago, 40 days ago, plus the
// lifetime row (record() would write it, but here we seed history directly).
repository.saveAndFlush(new SourceDocCountEntity("s", NOW_HOUR, 7));
repository.saveAndFlush(new SourceDocCountEntity("s", NOW_HOUR - 24L * 10, 50));
repository.saveAndFlush(new SourceDocCountEntity("s", NOW_HOUR - 24L * 40, 100));
totalRepository.saveAndFlush(new SourceDocTotalEntity("s", 157));
DocStats stats = counter().statsFor(List.of("s")).get("s");
assertEquals(157, stats.total()); // lifetime, including the out-of-window 40-days-ago docs
assertEquals(7, stats.last24h());
assertEquals(57, stats.last30d());
// The daily series (fetched separately, per source) covers the same 30-day window.
List<Long> series = counter().dailySeriesFor("s");
assertEquals(DocStats.DAYS, series.size());
assertEquals(7L, series.get(DocStats.DAYS - 1)); // today
assertEquals(50L, series.get(DocStats.DAYS - 11)); // 10 days ago
assertEquals(57L, series.stream().mapToLong(Long::longValue).sum());
}
@Test
void pruneRetiresOutOfWindowBucketsButKeepsTheLifetimeTotal() {
repository.saveAndFlush(new SourceDocCountEntity("s", NOW_HOUR, 7)); // today
repository.saveAndFlush(new SourceDocCountEntity("s", NOW_HOUR - 24L * 40, 100)); // 40d ago
totalRepository.saveAndFlush(new SourceDocTotalEntity("s", 107));
counter().pruneOldBuckets();
assertEquals(1, repository.count()); // only the in-window bucket remains
DocStats stats = counter().statsFor(List.of("s")).get("s");
assertEquals(107, stats.total()); // lifetime survives pruning
assertEquals(7, stats.last24h());
assertEquals(7, stats.last30d());
}
@Test
void aSourceWithNoRecordedDocsIsZero() {
assertEquals(DocStats.ZERO, counter().statsFor(List.of("unknown")).get("unknown"));
}
@SpringBootConfiguration
@AutoConfigurationPackage
static class TestApp {}
}
@@ -50,12 +50,7 @@ class SourceControllerTest {
SourceAccessGuard sourceGuard = new SourceAccessGuard(userService, properties, authority);
PolicyAccessGuard policyGuard = new PolicyAccessGuard(userService, properties, authority);
SourceOverviewService overviewService =
new SourceOverviewService(
sourceStore,
policyStore,
sourceGuard,
policyGuard,
new InProcessSourceDocCounter());
new SourceOverviewService(sourceStore, policyStore, sourceGuard, policyGuard);
triggerManager = mock(PolicyTriggerManager.class);
// A permissive input source so config validation passes and save can be exercised.
InputSource folderInput = mock(InputSource.class);
@@ -1,6 +1,7 @@
package stirling.software.proprietary.policy.source;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@@ -30,7 +31,6 @@ class SourceOverviewServiceTest {
private final SourceStore sourceStore = new InProcessSourceStore();
private final PolicyStore policyStore = new InProcessPolicyStore();
private final SourceDocCounter docCounter = new InProcessSourceDocCounter();
private SourceOverviewService service;
@BeforeEach
@@ -41,9 +41,7 @@ class SourceOverviewServiceTest {
PolicyManagementAuthority authority = mock(PolicyManagementAuthority.class);
SourceAccessGuard sourceGuard = new SourceAccessGuard(userService, properties, authority);
PolicyAccessGuard policyGuard = new PolicyAccessGuard(userService, properties, authority);
service =
new SourceOverviewService(
sourceStore, policyStore, sourceGuard, policyGuard, docCounter);
service = new SourceOverviewService(sourceStore, policyStore, sourceGuard, policyGuard);
}
@Test
@@ -113,8 +111,7 @@ class SourceOverviewServiceTest {
SourceAccessGuard sourceGuard = new SourceAccessGuard(userService, properties, authority);
PolicyAccessGuard policyGuard = new PolicyAccessGuard(userService, properties, authority);
SourceOverviewService scoped =
new SourceOverviewService(
sourceStore, policyStore, sourceGuard, policyGuard, docCounter);
new SourceOverviewService(sourceStore, policyStore, sourceGuard, policyGuard);
Source ours = teamSource("Ours", "/ours", 1L);
teamSource("Theirs", "/theirs", 2L);
@@ -131,19 +128,9 @@ class SourceOverviewServiceTest {
}
@Test
void documentCountsReflectRecordedDocs() {
void documentVolumeIsNotTrackedYet() {
Source a = source("A", "/a");
Source b = source("B", "/b");
docCounter.record(a.id(), 5);
docCounter.record(a.id(), 3);
SourceView av = find(service.overview(), a.id());
assertEquals(8, av.docsTotal());
assertEquals(8, av.docs24h());
assertEquals(8, av.docs30d());
// A source with no recorded documents reads as zero, not null.
assertEquals(0, find(service.overview(), b.id()).docsTotal());
assertNull(find(service.overview(), a.id()).docsTotal());
}
private Source source(String name, String directory) {
@@ -28,7 +28,6 @@ import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.AuthenticationType;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
@@ -59,7 +58,6 @@ class AuthControllerLoginTest {
@Mock private MfaService mfaService;
@Mock private TotpService totpService;
@Mock private RefreshRateLimitService refreshRateLimitService;
@Mock private ResourceAccessService resourceAccessService;
@BeforeEach
void setUp() {
@@ -82,8 +80,7 @@ class AuthControllerLoginTest {
refreshRateLimitService,
securityProperties,
applicationProperties,
new stirling.software.proprietary.service.AiUserDataService(null),
resourceAccessService);
new stirling.software.proprietary.service.AiUserDataService(null));
mockMvc = MockMvcBuilders.standaloneSetup(controller).build();
}
@@ -28,7 +28,6 @@ import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.enumeration.Role;
import stirling.software.proprietary.access.service.ResourceAccessService;
import stirling.software.proprietary.security.model.AuthenticationType;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.User;
@@ -60,7 +59,6 @@ class AuthControllerMoreTest {
@Mock private MfaService mfaService;
@Mock private TotpService totpService;
@Mock private RefreshRateLimitService refreshRateLimitService;
@Mock private ResourceAccessService resourceAccessService;
@BeforeEach
void setUp() {
@@ -83,8 +81,7 @@ class AuthControllerMoreTest {
refreshRateLimitService,
securityProperties,
applicationProperties,
new stirling.software.proprietary.service.AiUserDataService(null),
resourceAccessService);
new stirling.software.proprietary.service.AiUserDataService(null));
mockMvc = MockMvcBuilders.standaloneSetup(controller).build();
}
@@ -2,7 +2,6 @@ package stirling.software.proprietary.service;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
@@ -230,57 +229,6 @@ class AiWorkflowServiceTest {
// 1:1 mapping preserves each input's filename.
assertEquals("a.pdf", result.getResultFiles().get(0).getFileName());
assertEquals("b.pdf", result.getResultFiles().get(1).getFileName());
// Each output points back at the input it came from so the client versions it in place.
assertEquals(0, result.getResultFiles().get(0).getSourceIndex());
assertEquals(1, result.getResultFiles().get(1).getSourceIndex());
}
@Test
void mergeOutputHasNoSourceIndex() throws IOException {
MockMultipartFile a = pdf("a.pdf", "a-bytes");
MockMultipartFile b = pdf("b.pdf", "b-bytes");
stubOrchestrator(
"""
{"outcome":"tool_call","tool":"%s","parameters":{},"rationale":"Merging"}
"""
.formatted(MERGE_ENDPOINT));
when(toolMetadataService.isMultiInput(MERGE_ENDPOINT)).thenReturn(true);
when(toolMetadataService.shouldUnpackZipResponse(MERGE_ENDPOINT)).thenReturn(false);
stubEndpoint(MERGE_ENDPOINT, pdfResource("merged-bytes", "merged.pdf"));
stubFileStorage();
AiWorkflowResponse result =
service.orchestrate(requestFor(new MockMultipartFile[] {a, b}, "merge these"));
// A merge draws on several inputs, so there is no single source to version in place.
assertNull(result.getResultFiles().get(0).getSourceIndex());
}
@Test
void splitOutputsHaveNoSourceIndex() throws IOException {
MockMultipartFile input = pdf("doc.pdf", "original");
stubOrchestrator(
"""
{"outcome":"tool_call","tool":"%s","parameters":{},"rationale":"Splitting"}
"""
.formatted(SPLIT_ENDPOINT));
when(toolMetadataService.isMultiInput(SPLIT_ENDPOINT)).thenReturn(false);
when(toolMetadataService.shouldUnpackZipResponse(SPLIT_ENDPOINT)).thenReturn(true);
stubEndpoint(
SPLIT_ENDPOINT,
zipResource(
"doc.zip",
List.of(
new ZipEntryBytes("page-1.pdf", "page-one"),
new ZipEntryBytes("page-2.pdf", "page-two"))));
stubFileStorage();
AiWorkflowResponse result = service.orchestrate(requestFor(input, "split"));
// One input fanned out to many outputs, so none is a clean 1:1 version — the client adds
// them as fresh files and leaves the original in place.
assertNull(result.getResultFiles().get(0).getSourceIndex());
assertNull(result.getResultFiles().get(1).getSourceIndex());
}
@Test
@@ -1,26 +0,0 @@
-- Per-source document throughput, in two tables:
--
-- policy_source_doc_counts one row per source per hour bucket (hours-since-epoch), holding how
-- many documents that source fed into runs in that hour. Feeds the
-- rolling last-24h / last-30d windows and the 30-day daily series, and
-- is pruned to that window so it stays bounded.
-- policy_source_doc_totals a denormalized lifetime total per source, incremented alongside the
-- hourly bucket, so the overview reads the all-time figure in one row
-- instead of scanning a source's whole bucket history - and so the
-- hourly buckets can be pruned without losing it.
--
-- Gated by policies.enabled like the rest of the subsystem; Hibernate ddl-auto would also create
-- these, but the migration keeps the schema explicit for the Flyway-managed deployments.
CREATE TABLE IF NOT EXISTS policy_source_doc_counts (
source_id VARCHAR(255) NOT NULL,
bucket_hour BIGINT NOT NULL,
doc_count BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (source_id, bucket_hour)
);
CREATE TABLE IF NOT EXISTS policy_source_doc_totals (
source_id VARCHAR(255) NOT NULL,
doc_total BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (source_id)
);
@@ -1,14 +0,0 @@
-- Per-team classification taxonomy (gated by policies.enabled): the admin-editable vocabulary the
-- document classifier runs against. One row per team; the whole taxonomy lives as JSON in
-- taxonomy_json (authoritative on read). team_id is the natural key and a plain value (not a foreign
-- key) to stay decoupled from the security entities, so classification can be enabled or disabled
-- without touching them; the sentinel 0 holds the unteamed (login-disabled) taxonomy. Hibernate
-- ddl-auto would also create this, but this keeps the schema explicit for the Flyway-managed
-- deployments.
CREATE TABLE IF NOT EXISTS classification_taxonomies (
team_id BIGINT PRIMARY KEY,
taxonomy_json TEXT,
updated_at TIMESTAMP,
updated_by VARCHAR(255)
);
@@ -1,21 +0,0 @@
-- Resource access grants: which user/team may use a gated resource (portal, integration config).
CREATE TABLE IF NOT EXISTS resource_grants (
resource_grant_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
resource_type VARCHAR(64) NOT NULL,
resource_id VARCHAR(255) NOT NULL DEFAULT '',
principal_type VARCHAR(32) NOT NULL,
principal_id BIGINT NOT NULL,
permission VARCHAR(32) NOT NULL,
granted_by_user_id BIGINT,
created_at TIMESTAMP NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX IF NOT EXISTS uk_resource_grant
ON resource_grants (resource_type, resource_id, principal_type, principal_id, permission);
CREATE INDEX IF NOT EXISTS idx_resource_grants_lookup
ON resource_grants (resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_resource_grants_principal
ON resource_grants (principal_type, principal_id);
@@ -1,25 +0,0 @@
-- S3/MCP/API integration configs; config_encrypted holds an AES-GCM encrypted JSON blob.
CREATE TABLE IF NOT EXISTS integration_configs (
integration_config_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
integration_type VARCHAR(32) NOT NULL,
name VARCHAR(255) NOT NULL,
scope VARCHAR(32) NOT NULL,
owner_user_id BIGINT,
owner_team_id BIGINT,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
locked BOOLEAN NOT NULL DEFAULT FALSE,
default_access VARCHAR(32) NOT NULL DEFAULT 'EXPLICIT_ONLY',
config_encrypted TEXT,
created_at TIMESTAMP NOT NULL DEFAULT now(),
updated_at TIMESTAMP NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_integration_configs_owner
ON integration_configs (owner_user_id);
CREATE INDEX IF NOT EXISTS idx_integration_configs_type
ON integration_configs (integration_type);
CREATE INDEX IF NOT EXISTS idx_integration_configs_scope
ON integration_configs (scope);
+1 -1
View File
@@ -91,7 +91,7 @@ springBoot {
allprojects {
group = 'stirling.software'
version = '2.14.0'
version = '2.14.2'
configurations.configureEach {
exclude group: "org.springframework.boot", module: "spring-boot-starter-tomcat"
-2
View File
@@ -1,6 +1,5 @@
"""Agent modules for Stirling AI reasoning flows."""
from .document_classifier import DocumentClassifierAgent
from .execution import ExecutionPlanningAgent
from .orchestrator import OrchestratorAgent
from .pdf_create import PdfCreateAgent
@@ -10,7 +9,6 @@ from .pdf_review import PdfReviewAgent
from .user_spec import UserSpecAgent
__all__ = [
"DocumentClassifierAgent",
"ExecutionPlanningAgent",
"OrchestratorAgent",
"PdfCreateAgent",
@@ -1,569 +0,0 @@
{
"_generated": "AUTO-GENERATED from frontend/editor/src/proprietary/data/classificationTaxonomy.ts by editor/scripts/generate-classification-taxonomy.mts — do NOT edit by hand; run `task frontend:classifier-categories`.",
"categories": [
{
"id": "invoice",
"label": "Invoice",
"docTypes": [
{
"id": "invoice",
"label": "Invoice"
},
{
"id": "receipt",
"label": "Receipt"
},
{
"id": "credit_note",
"label": "Credit note"
},
{
"id": "purchase_order",
"label": "Purchase order"
},
{
"id": "quote",
"label": "Quote"
}
]
},
{
"id": "contract",
"label": "Contract",
"docTypes": [
{
"id": "nda",
"label": "Non-disclosure agreement"
},
{
"id": "employment_agreement",
"label": "Employment agreement"
},
{
"id": "service_agreement",
"label": "Service agreement"
},
{
"id": "lease_agreement",
"label": "Lease agreement"
},
{
"id": "master_service_agreement",
"label": "Master service agreement"
},
{
"id": "statement_of_work",
"label": "Statement of work"
},
{
"id": "terms_of_service",
"label": "Terms of service"
}
]
},
{
"id": "financial_statement",
"label": "Financial statement",
"docTypes": [
{
"id": "balance_sheet",
"label": "Balance sheet"
},
{
"id": "income_statement",
"label": "Income statement"
},
{
"id": "cash_flow_statement",
"label": "Cash flow statement"
},
{
"id": "bank_statement",
"label": "Bank statement"
},
{
"id": "annual_report",
"label": "Annual report"
}
]
},
{
"id": "report",
"label": "Report",
"docTypes": [
{
"id": "business_report",
"label": "Business report"
},
{
"id": "project_report",
"label": "Project report"
},
{
"id": "research_report",
"label": "Research report"
},
{
"id": "status_report",
"label": "Status report"
},
{
"id": "incident_report",
"label": "Incident report"
}
]
},
{
"id": "letter",
"label": "Letter",
"docTypes": [
{
"id": "business_letter",
"label": "Business letter"
},
{
"id": "cover_letter",
"label": "Cover letter"
},
{
"id": "recommendation_letter",
"label": "Recommendation letter"
},
{
"id": "complaint_letter",
"label": "Complaint letter"
},
{
"id": "demand_letter",
"label": "Demand letter"
}
]
},
{
"id": "form",
"label": "Form",
"docTypes": [
{
"id": "application_form",
"label": "Application form"
},
{
"id": "registration_form",
"label": "Registration form"
},
{
"id": "consent_form",
"label": "Consent form"
},
{
"id": "survey",
"label": "Survey"
},
{
"id": "questionnaire",
"label": "Questionnaire"
}
]
},
{
"id": "resume",
"label": "Resume",
"docTypes": [
{
"id": "resume",
"label": "Resume"
},
{
"id": "curriculum_vitae",
"label": "Curriculum vitae"
},
{
"id": "portfolio",
"label": "Portfolio"
},
{
"id": "reference_sheet",
"label": "Reference sheet"
}
]
},
{
"id": "tax_form",
"label": "Tax form",
"docTypes": [
{
"id": "tax_return",
"label": "Tax return"
},
{
"id": "w2",
"label": "W-2"
},
{
"id": "w9",
"label": "W-9"
},
{
"id": "form_1099",
"label": "Form 1099"
},
{
"id": "vat_return",
"label": "VAT return"
}
]
},
{
"id": "expense_report",
"label": "Expense report",
"docTypes": [
{
"id": "expense_report",
"label": "Expense report"
},
{
"id": "reimbursement_request",
"label": "Reimbursement request"
},
{
"id": "mileage_log",
"label": "Mileage log"
},
{
"id": "per_diem_claim",
"label": "Per diem claim"
}
]
},
{
"id": "presentation",
"label": "Presentation",
"docTypes": [
{
"id": "slide_deck",
"label": "Slide deck"
},
{
"id": "pitch_deck",
"label": "Pitch deck"
},
{
"id": "training_deck",
"label": "Training deck"
},
{
"id": "webinar_deck",
"label": "Webinar deck"
}
]
},
{
"id": "medical_record",
"label": "Medical record",
"docTypes": [
{
"id": "lab_result",
"label": "Lab result"
},
{
"id": "prescription",
"label": "Prescription"
},
{
"id": "discharge_summary",
"label": "Discharge summary"
},
{
"id": "medical_history",
"label": "Medical history"
},
{
"id": "imaging_report",
"label": "Imaging report"
},
{
"id": "vaccination_record",
"label": "Vaccination record"
}
]
},
{
"id": "legal_filing",
"label": "Legal filing",
"docTypes": [
{
"id": "court_filing",
"label": "Court filing"
},
{
"id": "complaint",
"label": "Complaint"
},
{
"id": "motion",
"label": "Motion"
},
{
"id": "subpoena",
"label": "Subpoena"
},
{
"id": "affidavit",
"label": "Affidavit"
},
{
"id": "deposition",
"label": "Deposition"
}
]
},
{
"id": "identity_document",
"label": "Identity document",
"docTypes": [
{
"id": "passport",
"label": "Passport"
},
{
"id": "drivers_license",
"label": "Driver's license"
},
{
"id": "national_id",
"label": "National ID"
},
{
"id": "birth_certificate",
"label": "Birth certificate"
},
{
"id": "visa",
"label": "Visa"
}
]
},
{
"id": "insurance",
"label": "Insurance",
"docTypes": [
{
"id": "insurance_policy",
"label": "Insurance policy"
},
{
"id": "insurance_claim",
"label": "Insurance claim"
},
{
"id": "certificate_of_insurance",
"label": "Certificate of insurance"
},
{
"id": "explanation_of_benefits",
"label": "Explanation of benefits"
}
]
},
{
"id": "real_estate",
"label": "Real estate",
"docTypes": [
{
"id": "deed",
"label": "Deed"
},
{
"id": "mortgage_agreement",
"label": "Mortgage agreement"
},
{
"id": "property_appraisal",
"label": "Property appraisal"
},
{
"id": "closing_disclosure",
"label": "Closing disclosure"
},
{
"id": "title_report",
"label": "Title report"
}
]
},
{
"id": "shipping",
"label": "Shipping",
"docTypes": [
{
"id": "bill_of_lading",
"label": "Bill of lading"
},
{
"id": "packing_slip",
"label": "Packing slip"
},
{
"id": "customs_declaration",
"label": "Customs declaration"
},
{
"id": "delivery_note",
"label": "Delivery note"
},
{
"id": "air_waybill",
"label": "Air waybill"
}
]
},
{
"id": "hr_document",
"label": "HR document",
"docTypes": [
{
"id": "offer_letter",
"label": "Offer letter"
},
{
"id": "performance_review",
"label": "Performance review"
},
{
"id": "payslip",
"label": "Payslip"
},
{
"id": "employee_handbook",
"label": "Employee handbook"
},
{
"id": "termination_letter",
"label": "Termination letter"
},
{
"id": "timesheet",
"label": "Timesheet"
}
]
},
{
"id": "academic_record",
"label": "Academic record",
"docTypes": [
{
"id": "transcript",
"label": "Transcript"
},
{
"id": "diploma",
"label": "Diploma"
},
{
"id": "certificate",
"label": "Certificate"
},
{
"id": "syllabus",
"label": "Syllabus"
},
{
"id": "thesis",
"label": "Thesis"
},
{
"id": "report_card",
"label": "Report card"
}
]
},
{
"id": "marketing_material",
"label": "Marketing material",
"docTypes": [
{
"id": "brochure",
"label": "Brochure"
},
{
"id": "flyer",
"label": "Flyer"
},
{
"id": "case_study",
"label": "Case study"
},
{
"id": "white_paper",
"label": "White paper"
},
{
"id": "press_release",
"label": "Press release"
}
]
},
{
"id": "technical_document",
"label": "Technical document",
"docTypes": [
{
"id": "user_manual",
"label": "User manual"
},
{
"id": "specification",
"label": "Specification"
},
{
"id": "api_documentation",
"label": "API documentation"
},
{
"id": "installation_guide",
"label": "Installation guide"
},
{
"id": "datasheet",
"label": "Datasheet"
},
{
"id": "release_notes",
"label": "Release notes"
}
]
}
],
"tags": [
"finance",
"legal",
"medical",
"hr",
"tax",
"insurance",
"marketing",
"technical",
"operations",
"academic",
"government",
"draft",
"final",
"signed",
"unsigned",
"executed",
"expired",
"amended",
"void",
"confidential",
"internal",
"public",
"pii",
"phi",
"certified",
"notarized",
"scanned",
"redacted",
"template",
"urgent"
]
}
@@ -1,193 +0,0 @@
from __future__ import annotations
import json
import logging
from pathlib import Path
from pydantic import Field
from pydantic_ai import Agent
from pydantic_ai.output import NativeOutput
from stirling.contracts import (
ClassificationTaxonomy,
ClassifyDocumentRequest,
ClassifyDocumentResponse,
DocumentClassificationResponse,
PageText,
)
from stirling.models import ApiModel
from stirling.services import AppRuntime
logger = logging.getLogger(__name__)
# Sentinel id for an answer that fell outside the supplied vocabulary.
UNKNOWN_LABEL = "unknown"
# Human-readable label shown for the off-list sentinel.
UNKNOWN_DISPLAY_LABEL = "Unknown"
# An off-list answer can never be reported as more confident than this, so a
# confident-but-wrong model answer can't clear an organisation's accept
# threshold downstream. See the design doc's "Validate" step.
UNKNOWN_MAX_CONFIDENCE = 0.2
# Pages read from each end of the document. A document's type is evident from
# its opening (and closing) pages, so a fixed window keeps cost and latency flat
# regardless of length. Promote to AppSettings if it ever needs tuning.
WINDOW_PAGES = 2
# The built-in vocabulary the classifier falls back to when a request doesn't
# supply its own. GENERATED from the TS source of truth
# (frontend/editor/src/proprietary/data/classificationTaxonomy.ts) via
# `task frontend:classifier-categories` — edit that file, not this JSON. Validated into the
# typed contract on import, so a malformed entry fails fast.
_DEFAULT_TAXONOMY_PATH = Path(__file__).with_name("default_classification_taxonomy.generated.json")
# The file carries an underscore-prefixed "_generated" notice (JSON has no
# comments); drop meta keys before validating against the strict contract.
_raw_taxonomy = json.loads(_DEFAULT_TAXONOMY_PATH.read_text(encoding="utf-8"))
DEFAULT_TAXONOMY = ClassificationTaxonomy.model_validate(
{key: value for key, value in _raw_taxonomy.items() if not key.startswith("_")}
)
_SYSTEM_PROMPT = (
"You identify what a document is, choosing only from a fixed vocabulary you "
"are given. Decide along three axes:\n"
"- category: the document's structural family. Choose EXACTLY ONE category id.\n"
"- doc_type: the specific instrument within that family. Choose EXACTLY ONE "
"doc_type id listed under the category you chose.\n"
"- tags: zero or more descriptor ids from the tag list.\n"
"\n"
"Rules:\n"
"- Use only ids from the supplied vocabulary. If nothing fits, return "
f'"{UNKNOWN_LABEL}" for category and/or doc_type.\n'
"- The doc_type you pick must belong to the category you pick.\n"
"- type_confidence (0.0-1.0) is how sure you are that doc_type is correct.\n"
"- Judge from the document's content and structure, not from keywords alone. "
"The document may be in any language.\n"
"- You are shown only the first and last pages; that is enough to identify the type."
)
class _ClassifierOutput(ApiModel):
"""Raw model answer, before it is validated against the taxonomy."""
category: str = Field(description="A category id from the vocabulary, or 'unknown'.")
doc_type: str = Field(description="A doc_type id belonging to the chosen category, or 'unknown'.")
type_confidence: float = Field(ge=0.0, le=1.0, description="Confidence that doc_type is correct.")
tags: list[str] = Field(default_factory=list, description="Descriptor ids drawn from the tag list.")
def render_taxonomy(taxonomy: ClassificationTaxonomy) -> str:
"""Render the vocabulary for the prompt, ids first so the model echoes them."""
lines = ["Categories (id (label): doc_types as id (label)):"]
for category in taxonomy.categories:
types = ", ".join(f"{doc_type.id} ({doc_type.label})" for doc_type in category.doc_types) or "(none)"
lines.append(f"- {category.id} ({category.label}): {types}")
lines.append(f"Tags: {', '.join(taxonomy.tags) or '(none)'}")
return "\n".join(lines)
def select_window(pages: list[PageText], window: int = WINDOW_PAGES) -> list[PageText]:
"""Return the first and last ``window`` pages, never overlapping.
Documents short enough that the two ends would meet are returned whole. The
caller usually sends just the window already; this is a defensive trim in
case it sends more.
"""
if window <= 0 or len(pages) <= window * 2:
return list(pages)
return [*pages[:window], *pages[-window:]]
def format_window(pages: list[PageText]) -> str:
if not pages:
return "(no extractable text)"
return "\n\n".join(f"[Page {page.page_number}]\n{page.text}" for page in pages)
def validate_against_taxonomy(
output: _ClassifierOutput,
taxonomy: ClassificationTaxonomy,
) -> DocumentClassificationResponse:
"""Coerce a raw model answer onto the supplied vocabulary.
An off-list category collapses both axes to ``unknown``; a doc_type that
isn't a child of its (valid) category collapses the type alone. Either
collapse caps confidence. Tags are filtered to the known set, de-duplicated,
and returned in the model's order. The model identifies; these rules decide
what is allowed to stand.
"""
categories_by_id = {category.id.lower(): category for category in taxonomy.categories}
allowed_tags = {tag.lower(): tag for tag in taxonomy.tags}
kept_tags: list[str] = []
for tag in output.tags:
canonical = allowed_tags.get(tag.strip().lower())
if canonical is not None and canonical not in kept_tags:
kept_tags.append(canonical)
category = categories_by_id.get(output.category.strip().lower())
if category is None:
return DocumentClassificationResponse(
category=UNKNOWN_LABEL,
category_label=UNKNOWN_DISPLAY_LABEL,
doc_type=UNKNOWN_LABEL,
doc_type_label=UNKNOWN_DISPLAY_LABEL,
type_confidence=min(output.type_confidence, UNKNOWN_MAX_CONFIDENCE),
tags=kept_tags,
)
types_by_id = {doc_type.id.lower(): doc_type for doc_type in category.doc_types}
doc_type = types_by_id.get(output.doc_type.strip().lower())
if doc_type is None:
return DocumentClassificationResponse(
category=category.id,
category_label=category.label,
doc_type=UNKNOWN_LABEL,
doc_type_label=UNKNOWN_DISPLAY_LABEL,
type_confidence=min(output.type_confidence, UNKNOWN_MAX_CONFIDENCE),
tags=kept_tags,
)
return DocumentClassificationResponse(
category=category.id,
category_label=category.label,
doc_type=doc_type.id,
doc_type_label=doc_type.label,
type_confidence=output.type_confidence,
tags=kept_tags,
)
class DocumentClassifierAgent:
"""Identifies a document's category, type, and tags against a taxonomy.
Reads the bounded page window supplied on the request (first/last
``WINDOW_PAGES``) and runs a single fast-model pass, then validates the
answer against the vocabulary so nothing off-list survives.
"""
def __init__(self, runtime: AppRuntime) -> None:
self.runtime = runtime
self._agent = Agent(
model=runtime.fast_model,
output_type=NativeOutput(_ClassifierOutput),
system_prompt=_SYSTEM_PROMPT,
model_settings=runtime.fast_model_settings,
)
async def classify(self, request: ClassifyDocumentRequest) -> ClassifyDocumentResponse:
# Override point: a request-supplied taxonomy (e.g. a future per-org / DB
# vocabulary the backend resolves) wins; the generated default is the fallback.
taxonomy = request.taxonomy or DEFAULT_TAXONOMY
window = select_window(request.pages)
prompt = self._build_prompt(request.file_name, taxonomy, window)
logger.debug("[classify] prompt:\n%s", prompt)
result = await self._agent.run(prompt)
return validate_against_taxonomy(result.output, taxonomy)
@staticmethod
def _build_prompt(file_name: str, taxonomy: ClassificationTaxonomy, window: list[PageText]) -> str:
return (
f"{render_taxonomy(taxonomy)}\n\n"
f"Document file name: {file_name}\n"
f"Document content (first and last pages):\n{format_window(window)}"
)
-4
View File
@@ -10,7 +10,6 @@ from pydantic_ai import Agent
from pydantic_ai.models.instrumented import InstrumentationSettings
from stirling.agents import (
DocumentClassifierAgent,
ExecutionPlanningAgent,
OrchestratorAgent,
PdfEditAgent,
@@ -25,7 +24,6 @@ from stirling.api.middleware import UserIdMiddleware
from stirling.api.routes import (
agent_capabilities_router,
agent_draft_router,
document_classifier_router,
document_router,
execution_router,
ledger_router,
@@ -97,7 +95,6 @@ async def lifespan(fast_api: FastAPI):
fast_api.state.execution_planning_agent = ExecutionPlanningAgent(runtime)
fast_api.state.math_auditor_agent = MathAuditorAgent(runtime)
fast_api.state.pdf_comment_agent = PdfCommentAgent(runtime)
fast_api.state.document_classifier_agent = DocumentClassifierAgent(runtime)
tracer_provider = setup_posthog_tracking(settings)
if tracer_provider:
Agent.instrument_all(InstrumentationSettings(tracer_provider=tracer_provider))
@@ -134,7 +131,6 @@ app.include_router(document_router, dependencies=_user_gate)
app.include_router(ledger_router, dependencies=_user_gate)
app.include_router(pdf_comments_router, dependencies=_user_gate)
app.include_router(agent_capabilities_router, dependencies=_user_gate)
app.include_router(document_classifier_router, dependencies=_user_gate)
@app.get("/health", response_model=HealthResponse)
-5
View File
@@ -5,7 +5,6 @@ from typing import Annotated
from fastapi import Depends, HTTPException, Request, status
from stirling.agents import (
DocumentClassifierAgent,
ExecutionPlanningAgent,
OrchestratorAgent,
PdfEditAgent,
@@ -56,10 +55,6 @@ def get_pdf_comment_agent(request: Request) -> PdfCommentAgent:
return request.app.state.pdf_comment_agent
def get_document_classifier_agent(request: Request) -> DocumentClassifierAgent:
return request.app.state.document_classifier_agent
def require_user_id() -> UserId:
"""FastAPI dependency for routes that touch per-user storage.
@@ -1,6 +1,5 @@
from .agent_capabilities import router as agent_capabilities_router
from .agent_drafts import router as agent_draft_router
from .document_classifier import router as document_classifier_router
from .documents import router as document_router
from .execution import router as execution_router
from .ledger import router as ledger_router
@@ -12,7 +11,6 @@ from .pdf_questions import router as pdf_question_router
__all__ = [
"agent_capabilities_router",
"agent_draft_router",
"document_classifier_router",
"document_router",
"execution_router",
"ledger_router",
@@ -1,24 +0,0 @@
from __future__ import annotations
from typing import Annotated
from fastapi import APIRouter, Depends
from stirling.agents import DocumentClassifierAgent
from stirling.api.dependencies import get_document_classifier_agent
from stirling.contracts import ClassifyDocumentRequest, ClassifyDocumentResponse
router = APIRouter(prefix="/api/v1/documents/classify", tags=["document-classifier"])
@router.post("", response_model=ClassifyDocumentResponse)
async def classify_document(
request: ClassifyDocumentRequest,
agent: Annotated[DocumentClassifierAgent, Depends(get_document_classifier_agent)],
) -> ClassifyDocumentResponse:
"""Classify a document from its supplied page text against the default taxonomy.
The caller sends the bounded page window inline, so no per-user document
storage is touched here — the request is self-contained.
"""
return await agent.classify(request)
-14
View File
@@ -36,14 +36,6 @@ from .contradiction import (
ContradictionReport,
ContradictionSeverity,
)
from .document_classifier import (
ClassificationTaxonomy,
ClassifyDocumentRequest,
ClassifyDocumentResponse,
DocumentCategory,
DocumentClassificationResponse,
DocumentType,
)
from .documents import (
DeleteDocumentResponse,
IngestDocumentRequest,
@@ -137,9 +129,6 @@ __all__ = [
"AiToolAgentStep",
"ArtifactKind",
"CannotContinueExecutionAction",
"ClassificationTaxonomy",
"ClassifyDocumentRequest",
"ClassifyDocumentResponse",
"Claim",
"CommentSpec",
"CompletedExecutionAction",
@@ -150,11 +139,8 @@ __all__ = [
"DeleteDocumentResponse",
"PurgeOwnerResponse",
"Discrepancy",
"DocumentCategory",
"DocumentClassificationResponse",
"DocumentMeta",
"DocumentSections",
"DocumentType",
"DiscrepancyKind",
"EditCannotDoResponse",
"EditClarificationRequest",
@@ -1,73 +0,0 @@
from __future__ import annotations
from pydantic import Field
from stirling.models import ApiModel
from .documents import PageText
class DocumentType(ApiModel):
"""A specific instrument within a category (e.g. ``nda`` inside ``contract``)."""
id: str = Field(min_length=1)
label: str = Field(min_length=1)
class DocumentCategory(ApiModel):
"""A structural family of documents, owning the doc_types shaped like it."""
id: str = Field(min_length=1)
label: str = Field(min_length=1)
doc_types: list[DocumentType] = Field(default_factory=list)
class ClassificationTaxonomy(ApiModel):
"""The vocabulary a document is classified against.
Supplied per request by the backend. When omitted, the engine falls back to
its small built-in default (see ``DEFAULT_TAXONOMY``). Tags are free-standing
descriptors that never own doc_types.
"""
categories: list[DocumentCategory] = Field(min_length=1)
tags: list[str] = Field(default_factory=list)
class ClassifyDocumentRequest(ApiModel):
"""Classify one document from its page text.
The caller sends the page text directly — typically just the bounded window
(first/last pages), since the classifier reads no more than that. There is no
ingestion or RAG step.
"""
file_name: str = Field(min_length=1)
pages: list[PageText] = Field(default_factory=list)
taxonomy: ClassificationTaxonomy | None = None
class DocumentClassificationResponse(ApiModel):
"""Terminal classification result.
``category`` and ``doc_type`` are ids drawn from the taxonomy (the internal
matching keys), or the sentinel ``"unknown"`` when the model's answer fell
outside it. ``category_label`` and ``doc_type_label`` are the human-readable
labels for those ids (what the UI shows); Python derives them from the
matched taxonomy entry so the two never drift. ``tags`` are the subset of the
model's tags that exist in the taxonomy. This is a plain answer from a
dedicated endpoint — it carries no ``outcome`` discriminator (it isn't one of
the orchestrator's WorkflowOutcome-routed union responses).
"""
category: str
category_label: str
doc_type: str
doc_type_label: str
type_confidence: float = Field(ge=0.0, le=1.0)
tags: list[str] = Field(default_factory=list)
# Only one response shape today; kept as a named alias so routes and agents have
# a stable response type to import.
ClassifyDocumentResponse = DocumentClassificationResponse
-177
View File
@@ -1,177 +0,0 @@
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
from stirling.agents.document_classifier import (
DEFAULT_TAXONOMY,
UNKNOWN_LABEL,
UNKNOWN_MAX_CONFIDENCE,
DocumentClassifierAgent,
_ClassifierOutput,
render_taxonomy,
select_window,
validate_against_taxonomy,
)
from stirling.contracts import (
ClassificationTaxonomy,
ClassifyDocumentRequest,
DocumentCategory,
DocumentClassificationResponse,
PageText,
)
from stirling.services.runtime import AppRuntime
def _page(number: int, text: str = "x") -> PageText:
return PageText(page_number=number, text=text)
# ── select_window ───────────────────────────────────────────────────────────
def test_select_window_returns_short_documents_whole() -> None:
pages = [_page(1), _page(2), _page(3), _page(4)]
assert select_window(pages, window=2) == pages
def test_select_window_takes_both_ends_without_overlap() -> None:
pages = [_page(n) for n in range(1, 6)] # 5 pages
selected = select_window(pages, window=2)
assert [p.page_number for p in selected] == [1, 2, 4, 5]
def test_select_window_handles_empty() -> None:
assert select_window([], window=2) == []
def test_select_window_zero_returns_all() -> None:
pages = [_page(1), _page(2), _page(3)]
assert select_window(pages, window=0) == pages
# ── validate_against_taxonomy ────────────────────────────────────────────────
def test_valid_classification_is_preserved() -> None:
output = _ClassifierOutput(category="contract", doc_type="nda", type_confidence=0.95, tags=["legal", "signed"])
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert isinstance(result, DocumentClassificationResponse)
assert result.category == "contract"
assert result.doc_type == "nda"
assert result.type_confidence == 0.95
assert result.tags == ["legal", "signed"]
def test_off_list_category_collapses_to_unknown_with_capped_confidence() -> None:
output = _ClassifierOutput(category="spaceship", doc_type="warp_core", type_confidence=0.99)
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.category == UNKNOWN_LABEL
assert result.doc_type == UNKNOWN_LABEL
assert result.type_confidence == UNKNOWN_MAX_CONFIDENCE
def test_off_list_type_keeps_category_but_unknown_type() -> None:
output = _ClassifierOutput(category="contract", doc_type="invoice", type_confidence=0.9)
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.category == "contract"
assert result.doc_type == UNKNOWN_LABEL
assert result.type_confidence == UNKNOWN_MAX_CONFIDENCE
def test_type_from_a_different_category_is_not_a_child() -> None:
# "lab_result" is a valid type, but only under medical_record, not contract.
output = _ClassifierOutput(category="contract", doc_type="lab_result", type_confidence=0.8)
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.category == "contract"
assert result.doc_type == UNKNOWN_LABEL
def test_matching_is_case_insensitive_and_returns_canonical_ids() -> None:
output = _ClassifierOutput(category="Contract", doc_type="NDA", type_confidence=0.7, tags=["LEGAL"])
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.category == "contract"
assert result.doc_type == "nda"
assert result.tags == ["legal"]
def test_unknown_tags_dropped_and_deduplicated_in_order() -> None:
output = _ClassifierOutput(
category="invoice",
doc_type="invoice",
type_confidence=0.9,
tags=["finance", "made-up", "finance", "legal"],
)
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.tags == ["finance", "legal"]
def test_low_confidence_is_not_raised_when_collapsing() -> None:
output = _ClassifierOutput(category="nope", doc_type="nope", type_confidence=0.05)
result = validate_against_taxonomy(output, DEFAULT_TAXONOMY)
assert result.type_confidence == 0.05 # min(0.05, 0.2)
# ── render_taxonomy ──────────────────────────────────────────────────────────
def test_render_taxonomy_lists_ids_and_tags() -> None:
rendered = render_taxonomy(DEFAULT_TAXONOMY)
assert "contract" in rendered
assert "nda" in rendered
assert "finance" in rendered
def test_render_taxonomy_handles_category_without_types() -> None:
taxonomy = ClassificationTaxonomy(
categories=[DocumentCategory(id="memo", label="Memo", doc_types=[])],
tags=[],
)
rendered = render_taxonomy(taxonomy)
assert "(none)" in rendered
# ── DocumentClassifierAgent (inline page text) ───────────────────────────────
@pytest.mark.anyio
async def test_classify_validates_model_output_against_default_taxonomy(runtime: AppRuntime) -> None:
agent = DocumentClassifierAgent(runtime)
agent._agent.run = AsyncMock(
return_value=SimpleNamespace(
output=_ClassifierOutput(category="invoice", doc_type="invoice", type_confidence=0.97, tags=["finance"])
)
)
result = await agent.classify(
ClassifyDocumentRequest(
file_name="invoice.pdf",
pages=[PageText(page_number=1, text="Invoice INV-1 total due 100.00")],
)
)
assert isinstance(result, DocumentClassificationResponse)
assert result.category == "invoice"
assert result.doc_type == "invoice"
assert result.tags == ["finance"]
@pytest.mark.anyio
async def test_classify_collapses_off_list_model_answer(runtime: AppRuntime) -> None:
agent = DocumentClassifierAgent(runtime)
agent._agent.run = AsyncMock(
return_value=SimpleNamespace(
output=_ClassifierOutput(category="boarding_pass", doc_type="seat", type_confidence=0.9)
)
)
result = await agent.classify(
ClassifyDocumentRequest(file_name="weird.pdf", pages=[PageText(page_number=1, text="Some text")])
)
assert isinstance(result, DocumentClassificationResponse)
assert result.category == UNKNOWN_LABEL
assert result.doc_type == UNKNOWN_LABEL
assert result.type_confidence == UNKNOWN_MAX_CONFIDENCE
@@ -1,73 +0,0 @@
from __future__ import annotations
from collections.abc import Iterator
import pytest
from fastapi.testclient import TestClient
from stirling.api import app
from stirling.api.dependencies import get_document_classifier_agent
from stirling.contracts import (
ClassifyDocumentRequest,
ClassifyDocumentResponse,
DocumentClassificationResponse,
)
class StubClassifierAgent:
"""Stands in for DocumentClassifierAgent so route tests don't call a model."""
def __init__(self, response: ClassifyDocumentResponse) -> None:
self._response = response
async def classify(self, _request: ClassifyDocumentRequest) -> ClassifyDocumentResponse:
return self._response
@pytest.fixture
def classification_client() -> Iterator[TestClient]:
app.dependency_overrides[get_document_classifier_agent] = lambda: StubClassifierAgent(
DocumentClassificationResponse(
category="contract",
category_label="Contract",
doc_type="nda",
doc_type_label="Non-disclosure agreement",
type_confidence=0.96,
tags=["legal", "signed"],
)
)
try:
yield TestClient(app)
finally:
app.dependency_overrides.pop(get_document_classifier_agent, None)
def test_classify_returns_camel_cased_result(classification_client: TestClient) -> None:
response = classification_client.post(
"/api/v1/documents/classify",
json={"fileName": "nda.pdf", "pages": [{"pageNumber": 1, "text": "Mutual NDA between A and B."}]},
)
assert response.status_code == 200
body = response.json()
assert body["category"] == "contract"
assert body["categoryLabel"] == "Contract"
assert body["docType"] == "nda"
assert body["docTypeLabel"] == "Non-disclosure agreement"
assert body["typeConfidence"] == 0.96
assert body["tags"] == ["legal", "signed"]
def test_classify_accepts_empty_pages(classification_client: TestClient) -> None:
response = classification_client.post(
"/api/v1/documents/classify",
json={"fileName": "blank.pdf", "pages": []},
)
assert response.status_code == 200
def test_classify_rejects_empty_file_name(classification_client: TestClient) -> None:
response = classification_client.post(
"/api/v1/documents/classify",
json={"fileName": "", "pages": []},
)
assert response.status_code == 422
@@ -3715,7 +3715,6 @@ backToFolder = "Back to {{folder}}"
backToMyFiles = "Back to My Files"
breadcrumbs = "Folder path"
cancel = "Cancel"
classification = "Classification"
clearSearch = "Clear search"
clearSelection = "Clear selection"
closeDetails = "Close details"
@@ -3873,14 +3872,12 @@ uploadFilesFailedDetail = "Could not upload files: {{message}}"
[filesPage.field]
added = "Added"
category = "Category"
confidence = "Confidence"
count = "Files"
folder = "Folder"
modified = "Modified"
name = "Name"
size = "Size"
tags = "Tags"
toolHistory = "Tool history"
toolHistoryAtVersion = "Cumulative tool chain"
totalSize = "Total size"
type = "Type"
@@ -5982,68 +5979,21 @@ ssn = "Social Security numbers"
[policies.sidebar]
activeCount = "{{count}} active"
infoAriaLabel = "What is a policy?"
infoTooltip = "A policy is a fixed set of tools that runs automatically whenever it's triggered — for example when a new document arrives — enforcing rules like redacting PII with no manual steps."
loading = "Loading…"
optionsAriaLabel = "Policy options"
policySettings = "Policy settings"
railAriaLabel = "{{label}} policy — {{status}}"
railSuffixActive = " (Active)"
railSuffixPaused = " (Paused)"
setUp = "Set up"
title = "Policies"
upgradeToEnterprise = "Upgrade to enterprise"
whatIsPolicy = "What is a policy?"
[policies.settings]
onExport = "On export"
onUpload = "On upload"
noneExport = "No policies currently run on export."
noneUpload = "No policies currently run on upload."
reorderHandle = "Drag to reorder"
runOrderDesc = "When more than one policy runs on the same trigger, they run in this order — each on the previous policy's output. Drag to reorder."
title = "Policy settings"
[policies.status]
active = "Active"
paused = "Paused"
setup = "Set up"
[policies.taxonomy]
add = "Add"
addCategory = "Add category"
addSub = "Add sub-category"
addTag = "Add a tag"
addTagPlaceholder = "Add a tag"
categories = "categories"
categoryLabel = "Category"
collapse = "Collapse"
customNote = "Customized for your team."
defaultNote = "Using the built-in default, shared with your team."
edit = "Edit taxonomy"
emptyCategories = "No categories yet — add one to get started."
expand = "Expand"
export = "Export JSON"
id = "ID"
import = "Import JSON"
importError = "Couldn't import that file."
managedNote = "The taxonomy is managed by your team leader."
modalSubtitle = "Shared with your whole team. Categories, their sub-categories, and tags the classifier uses."
modalTitle = "Classification taxonomy"
noTags = "No tags yet."
removeCategory = "Remove category"
removeSub = "Remove sub-category"
removeTag = "Remove {{tag}}"
resetToDefault = "Reset to default"
saveForTeam = "Save for team"
saving = "Saving…"
sectionLabel = "Classification taxonomy"
startFromScratch = "Start from scratch"
subCategories = "sub-categories"
subCount = "{{count}} sub"
subLabel = "Sub-category"
tags = "Tags"
view = "View taxonomy"
[policies.toolConfig]
enableAriaLabel = "Enable {{tool}}"
infoAriaLabel = "What does {{tool}} do?"
@@ -6984,6 +6934,21 @@ manage = "Manage"
description = "Policies and legal information for this service."
title = "Legal Documents"
[settings.licenses]
backendDescription = "Licenses for backend dependencies bundled with this server."
backendLabel = "Backend Licenses"
backendTitle = "Backend 3rd Party Licenses"
empty = "No dependencies found."
frontendDescription = "Licenses for frontend dependencies bundled into the release build."
frontendLabel = "Frontend Licenses"
frontendTitle = "Frontend 3rd Party Licenses"
license = "License"
listDescription = "The list is shown directly in the UI from the release bundle or backend endpoint."
listTitle = "Bundled dependencies"
loadError = "Failed to load third-party licenses"
module = "Module"
version = "Version"
[settings.licensingAnalytics]
audit = "Audit"
plan = "Plan"

Some files were not shown because too many files have changed in this diff Show More