Compare commits

...
Author SHA1 Message Date
EthanHealy01andGitHub ab1874b39f Merge branch 'main' into feature/policiesVisualiserComponent 2026-07-14 12:18:15 +01:00
James BruntonandGitHub 776749277c Redesign policies to use typed mappings properly (#7017)
# Description of Changes
The Policies page and all the frontend logic for running Policies is not
making use of the bidirectional type mappings that we now have to safely
convert from frontend to backend param models and vice versa. This
changes the way we track the types throughout so we use the mappings
properly.

Because of this, the Add Watermark settings in Policies now actually
pre-populate with the defaults instead of with nothing like they
previously did.

<img width="791" height="725" alt="image"
src="https://github.com/user-attachments/assets/cbdf4ae0-35af-4792-bf64-89216e48d304"
/>
2026-07-14 09:58:04 +00:00
James BruntonandGitHub 41b1b89fcb Fix Policies page showing the Editor as a source twice (#7022)
# Description of Changes
The Policies page currently hard-codes the Editor to be available as a
source, but we now also have a virtual Editor source on the backend,
which the Policies page also renders. This removes the now-unnecessary
hard-coded Editor source.

## Before

<img width="842" height="640" alt="image"
src="https://github.com/user-attachments/assets/d78b33a3-fed4-4bb0-a02f-489ca2ae0614"
/>

## After

<img width="785" height="586" alt="image"
src="https://github.com/user-attachments/assets/37aa664f-74f2-41c7-b89a-9b483bffc3a2"
/>
2026-07-14 09:43:07 +00:00
James BruntonandGitHub 4d4e994562 Fix crash in Processor when loading tool settings with tooltips (#7015)
# Description of Changes
Some of the tool settings make use of editor preferences indirectly, but
the Processor never gets that provider, so it crashes when trying to
load them.
2026-07-14 09:09:23 +00:00
2b118556f3 Merge hotfix/v2.14.2 into main (#7023)
Merges the `hotfix/v2.14.2` branch into `main`.
on the hotfix branch:

### What this actually changes on `main`
- **Version bump 2.14.1 → 2.14.2** `build.gradle`, `tauri.conf.json`,
both AUR `PKGBUILD`s, and the two `serverExperienceSimulations.ts`
test-config files.
- **Fix Postgres user settings for some users** removes `@Lob` from
`User.java that broke settings for some Postgres users.
- **Release workflow: stop msiexec hang in Windows signature verify**

---------

Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
Co-authored-by: LFdev <146497073+LFd3v@users.noreply.github.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-07-13 23:45:19 +01:00
Anthony Stirling fbaff56d1c Merge hotfix/v2.14.2 into main (v2.14.2 bump, Postgres user settings fix, msiexec release fix) 2026-07-13 20:23:14 +01:00
EthanHealy01andGitHub fc210bc761 Merge branch 'main' into feature/policiesVisualiserComponent 2026-07-13 18:45:05 +01:00
EthanHealy01 e3802c56b2 fix colors 2026-07-13 17:37:13 +01:00
Anthony StirlingandGitHub a1b15e0570 Portal: dark disabled buttons and role column width (#7004)
# Description of Changes

Fixes disabled buttons rendering as plain grey in dark mode, and widens
the Users role column so "Organisation Owner" no longer clips.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after

<!-- paste before / after screenshots here -->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-13 15:00:01 +00:00
James BruntonandGitHub 76549288a9 Redesign S3 connections to use connection resolver (#6965)
# Description of Changes
Redesign S3 connections based on feedback from #6948. Also redesigns the
UI for Sources to make them more like the Pipelines page which improves
UX quite a bit. There's still plenty more UI/UX work for Sources and S3
but moving in the right direction.
2026-07-13 14:44:41 +00:00
Anthony StirlingandGitHub b5d0c4a5ed Portal Home: SVG quick-action icons (#6998)
# Description of Changes

Replaces the ASCII quick-action glyphs on the Home hero with crisp
stroke SVG icons.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after
<img width="2880" height="2726" alt="after-home-dark"
src="https://github.com/user-attachments/assets/ab4c3611-25a6-4b2c-a993-99ce2f7b7558"
/>
<img width="2880" height="2726" alt="after-home-light"
src="https://github.com/user-attachments/assets/b0b342b9-4436-4791-b0a6-92837c3ec355"
/>
<img width="2880" height="2726" alt="before-home-dark"
src="https://github.com/user-attachments/assets/93d73393-5a81-4f34-a483-90671a6ae79e"
/>
<img width="2880" height="2726" alt="before-home-light"
src="https://github.com/user-attachments/assets/119389cc-78ef-4a43-9f9a-d551b03c9733"
/>


---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-13 13:26:07 +00:00
Anthony StirlingandGitHub 8bfcf6eb7e Portal: theme-aware code blocks and hero-navy token (#7003)
# Description of Changes

Makes the code-snippet boxes theme-aware (a light palette in light mode)
and moves the hero navy into a design token without changing the colour
itself.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after

<img width="2136" height="272" alt="after-codeblock-light"
src="https://github.com/user-attachments/assets/ebdd4a7d-2d1a-429a-971b-0b04e93854fe"
/>
<img width="1800" height="740" alt="before-codeblock-light"
src="https://github.com/user-attachments/assets/6819231a-10d5-4730-9b7d-3c36dc1c8170"
/>

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-13 13:26:00 +00:00
James BruntonandGitHub b019f9b570 Fix missing and broken translations in Processor (#7016)
# Description of Changes
<img width="385" height="92" alt="image"
src="https://github.com/user-attachments/assets/7f4b1921-72e8-4c18-a4de-4b9736a5a2f1"
/>

Started from trying to fix this, but became a larger piece of work to
find missing/broken translations in the Processor and fix as many as I
could.
2026-07-13 12:55:36 +00:00
EthanHealy01 c59571f17c pre commit fix and other fixes 2026-07-13 11:40:41 +01:00
EthanHealy01 7cdf72c5f9 break flow into subfiles, it was massive 2026-07-13 02:17:52 +01:00
Anthony Stirling 38ccea074c Version bump 2026-07-08 10:50:37 +01:00
Anthony Stirling a7307ff393 Fix Postgres user settings for some users 2026-07-08 10:50:36 +01:00
Anthony Stirling 72729e99c1 fix(release): stop msiexec hang in Windows signature verify; don't force latest or regen release notes 2026-07-07 23:35:12 +01:00
Anthony Stirling 5fba2720f0 Fix cert sign not showing under certain instances (#6908) 2026-07-07 22:45:56 +01:00
01a1ef8c44 Fix missing app icon on Linux/Wayland (#6875)
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-07-07 22:15:01 +01:00
LudyandAnthony Stirling 8535c7e9ac feat(ui): add dedicated third-party license sections to settings (#6820) 2026-07-07 22:15:01 +01:00
ConnorYohandAnthony Stirling cca3f42623 Set App version to v2.14.1 (#6891)
Upped version in build.gradle then ran build so version falls through
2026-07-07 12:15:01 +01:00
James BruntonandAnthony Stirling 1b7ffcdbac Fix tooltip positioning on Add Page Numbers (#6885)
# Description of Changes
## Before

<img width="483" height="227" alt="image"
src="https://github.com/user-attachments/assets/4bf86eec-a9cc-4f63-84f0-4eb2bd535bab"
/>

## After

<img width="732" height="235" alt="image"
src="https://github.com/user-attachments/assets/101d2ea4-36e8-4e8f-990a-d72b33fa0ac2"
/>
2026-07-07 12:09:51 +01:00
LudyandAnthony Stirling 67a0ca6110 fix(frontend): respect analytics config before initializing PostHog (#6812)
# Description of Changes

Please provide a summary of the changes, including:

- What was changed
- Moved PostHog startup out of `index.tsx` and into a config-aware
initializer inside `AppProviders`.
- Added a dedicated `usePosthogTracking` hook that only initializes
PostHog when `enableAnalytics` is explicitly `true` and `enablePosthog`
is not disabled.
- Kept cookie-consent handling in the same flow so consent is applied
only after PostHog is actually initialized.
- Removed the unconditional `PostHogProvider` and `posthog.init(...)`
bootstrap from the app entrypoint.
- Added targeted frontend tests covering analytics-disabled and
analytics-enabled startup behavior.

- Why the change was made
- The previous frontend bootstrap initialized PostHog before app config
was loaded, so disabling analytics in the UI or via environment settings
did not prevent PostHog network activity.
- This change makes analytics behavior follow the server-provided config
instead of always connecting on page load.

Closes #6358

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-07 12:09:51 +01:00
Anthony Stirling 8e4b2e2fc6 Disable update check and notification in SaaS mode (#6863)
# Description of Changes

In SaaS mode the self-hosted "Update Available" notification could still
appear and the update-check code (external call to
`supabase.stirling.com/functions/v1/updates`) still ran, even though the
cloud owns app versioning. The web `UpdateStartupPopup` was already
SaaS-gated via a null override, but two other paths were not:

- **Desktop app in SaaS connection mode** - `useDesktopUpdatePopup()`
ran its startup check and rendered the `UpdateModal` regardless of
connection mode, so a self-hosted update popup appeared while connected
to SaaS.
- **Settings → General** - the core `GeneralSection` fired
`checkForUpdate()` on mount unconditionally, even when the update
section was hidden (as SaaS does), so the external call still ran.

**What changed**

- `useDesktopUpdatePopup.ts` - the startup timer now bails out
immediately when `connectionModeService.getCurrentMode() === "saas"`. No
mode lookup, no external fetch, no modal.
- `core/GeneralSection.tsx` - the mount `checkForUpdate()` now returns
early when `hideUpdateSection` is set, so hiding the section (web SaaS,
managed-disabled desktop) also stops the external call.
- `desktop/GeneralSection.tsx` - passes `hideUpdateSection` when
`useSaaSMode()` is true, which (via the above) suppresses the settings
check in desktop-SaaS too.

**Why** - in SaaS the update check should never be called and no update
notification should be shown; the cloud handles versioning.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-07 12:09:51 +01:00
Anthony Stirling 3c93457021 Fix rearrange-pages DUPLICATE producing shared page nodes (pypdf cyclic-references CI break) (#6851) 2026-07-07 12:09:51 +01:00
116 changed files with 5986 additions and 1932 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.14.1
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.14.1
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
+23 -6
View File
@@ -510,6 +510,7 @@ jobs:
# cargo output unsigned, so checking it produces false negatives.
- name: Verify Windows Code Signature
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
timeout-minutes: 15
shell: pwsh
run: |
$allSigned = $true
@@ -531,11 +532,26 @@ jobs:
# Extract MSI and verify the inner exe (the file that actually gets installed).
# This is the critical check - AV flags the installed exe at runtime.
# Use lessmsi, not `msiexec /a`: msiexec serializes on the global
# _MSIExecute mutex and hangs forever on hosted runners when another
# installer is busy. lessmsi reads MSI tables directly - no mutex, no service.
$msi = $msiFiles[0].FullName
$extractDir = Join-Path $env:RUNNER_TEMP "msi-verify"
if (Test-Path $extractDir) { Remove-Item $extractDir -Recurse -Force }
$proc = Start-Process msiexec.exe -ArgumentList '/a', $msi, '/qn', "TARGETDIR=$extractDir" -Wait -PassThru -NoNewWindow
if ($proc.ExitCode -eq 0) {
New-Item -ItemType Directory -Force -Path $extractDir | Out-Null
choco install lessmsi -y --no-progress --limit-output | Out-Null
# Bound the extraction and kill on hang (defence in depth over timeout-minutes).
$proc = Start-Process lessmsi -ArgumentList 'x', "`"$msi`"", "`"$extractDir\`"" -PassThru -NoNewWindow
if (-not $proc.WaitForExit(120000)) {
try { $proc.Kill() } catch {}
Write-Host "[ERROR] MSI extraction timed out after 120s"
$allSigned = $false
} elseif ($proc.ExitCode -ne 0) {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
} else {
$innerExe = Get-ChildItem -Path $extractDir -Filter "stirling-pdf.exe" -Recurse -File | Select-Object -First 1
if ($innerExe) {
$sig = Get-AuthenticodeSignature -FilePath $innerExe.FullName
@@ -548,9 +564,6 @@ jobs:
Write-Host "[ERROR] Could not find stirling-pdf.exe inside MSI"
$allSigned = $false
}
} else {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
}
if (-not $allSigned) {
@@ -800,7 +813,11 @@ jobs:
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
# Don't regenerate/append notes on re-runs, and don't force this into the
# "Latest" slot - leave the release body and latest marker as they are.
generate_release_notes: false
append_body: false
make_latest: false
fail_on_unmatched_files: true
# Installers + updater payloads + manifest. .sig contents are embedded
# in latest.json so the .sig files themselves are not uploaded.
@@ -43,6 +43,10 @@ public class IntegrationConfigService {
private final OwnershipService ownership;
private final SecretMasker secretMasker;
private final ResourceGrantRepository grantRepository;
// Bean-discovered extension points: features that understand a type contribute its config
// schema and report what still references a config, without this module depending on them.
private final List<IntegrationConfigValidator> validators;
private final List<IntegrationConfigUsageCheck> usageChecks;
// ---- commands ----
@@ -66,13 +70,21 @@ public class IntegrationConfigService {
? DefaultAccessPolicy.EXPLICIT_ONLY
: request.defaultAccess());
// TEAM scope may omit the team id: default to the caller's own team so clients (the
// portal) need not know it. assignOwnership still enforces admin-or-leader of that team.
Long ownerTeamId = request.ownerTeamId();
if (ownerTeamId == null && scope == OwnerScope.TEAM && currentUser.getTeam() != null) {
ownerTeamId = currentUser.getTeam().getId();
}
ownership.assignOwnership(
cfg,
scope,
request.ownerTeamId(),
ownerTeamId,
currentUser,
() -> lockedServerExists(cfg.getIntegrationType()));
cfg.setConfig(writeJson(secretMasker.sanitize(request.config())));
Map<String, Object> config = secretMasker.sanitize(request.config());
validateConfig(cfg.getIntegrationType(), config);
cfg.setConfig(writeJson(config));
return repository.save(cfg);
}
@@ -101,8 +113,10 @@ public class IntegrationConfigService {
cfg.setDefaultAccess(request.defaultAccess());
}
if (request.config() != null) {
cfg.setConfig(
writeJson(secretMasker.merge(readJson(cfg.getConfig()), request.config())));
Map<String, Object> merged =
secretMasker.merge(readJson(cfg.getConfig()), request.config());
validateConfig(cfg.getIntegrationType(), merged);
cfg.setConfig(writeJson(merged));
}
return repository.save(cfg);
}
@@ -113,6 +127,15 @@ public class IntegrationConfigService {
if (!ownership.canManage(TYPE, cfg, currentUser)) {
throw forbidden("You cannot manage this integration");
}
// Refuse to pull a connection out from under whatever still references it.
List<String> usages =
usageChecks.stream()
.flatMap(check -> check.usagesOf(cfg.getId()).stream())
.toList();
if (!usages.isEmpty()) {
throw new ResponseStatusException(
HttpStatus.CONFLICT, "Integration is in use by: " + String.join(", ", usages));
}
// Drop grants sharing this config so they do not dangle as dead rows.
grantRepository.deleteByResourceTypeAndResourceId(TYPE, String.valueOf(cfg.getId()));
repository.delete(cfg);
@@ -188,6 +211,19 @@ public class IntegrationConfigService {
// ---- integration-specific glue ----
/** Runs every registered validator for the type; unknown types save free-form. */
private void validateConfig(IntegrationType type, Map<String, Object> config) {
for (IntegrationConfigValidator validator : validators) {
if (validator.type() == type) {
try {
validator.validate(config == null ? Map.of() : config);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
}
}
}
/** A non-admin can't create a personal config of a type an admin has locked at server scope. */
private boolean lockedServerExists(IntegrationType type) {
return repository.findByScope(OwnerScope.SERVER).stream()
@@ -0,0 +1,15 @@
package stirling.software.proprietary.integration.service;
import java.util.List;
/**
* Reports what still references an integration config, so deletion can be refused instead of
* pulling a connection out from under a live consumer. Implementations are beans discovered by
* {@link IntegrationConfigService} (e.g. the policy subsystem reporting sources and pipelines that
* reference a connection).
*/
public interface IntegrationConfigUsageCheck {
/** Human-readable labels of everything still using the config; empty when unreferenced. */
List<String> usagesOf(long configId);
}
@@ -0,0 +1,23 @@
package stirling.software.proprietary.integration.service;
import java.util.Map;
import stirling.software.proprietary.integration.model.IntegrationType;
/**
* Validates one integration type's config map at save time. Implementations are beans discovered by
* {@link IntegrationConfigService}, so the feature that understands a type (e.g. the policy S3
* backend) owns its schema without the integration module depending on it. Types with no registered
* validator save free-form.
*/
public interface IntegrationConfigValidator {
/** The type this validator understands. */
IntegrationType type();
/**
* Validates the config as it will be stored (secrets already sanitized/merged, so values are
* real, never the redaction mask). Throws {@link IllegalArgumentException} on bad config.
*/
void validate(Map<String, Object> config);
}
@@ -120,6 +120,7 @@ public class PolicyController {
throws IOException {
stampPolicyAudit(definition);
requireRunnable(definition);
validateAdHocOutput(definition);
PolicyInputs inputs = toInputs(files);
PolicyRunHandle handle =
policyRunner.runAdHoc(definition, inputs, PolicyProgressListener.NOOP);
@@ -140,6 +141,7 @@ public class PolicyController {
throws IOException {
stampPolicyAudit(definition);
requireRunnable(definition);
validateAdHocOutput(definition);
PolicyInputs inputs = toInputs(files);
SseEmitter emitter =
@@ -530,6 +532,24 @@ public class PolicyController {
}
}
/**
* Authorization-check an ad-hoc run's output while the caller's principal is present (this
* request thread). The worker thread that later delivers carries no security context, so an S3
* output's connection-access check would be skipped there; without this gate a caller could
* reference another tenant's connection by id and write to it (confused deputy). Stored
* policies are covered by save-time {@link PolicyValidator#validate} instead.
*/
private void validateAdHocOutput(PipelineDefinition definition) {
if (definition.output() == null) {
return;
}
try {
policyValidator.validateOutput(definition.output());
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
}
/**
* Ad-hoc runs (AI / one-off pipelines) are still editor activity, so their supplied documents
* feed the same virtual editor source as stored editor policies, counted against the caller's
@@ -52,7 +52,19 @@ public class PolicyValidator {
InputSpec spec = source.toInputSpec();
inputSourceFor(spec).validate(spec);
}
outputSinkFor(policy.output()).validate(policy.output());
validateOutput(policy.output());
}
/**
* Validate an output spec against its sink. Must be called on a request thread (caller's
* principal present) so an S3 output's connection is authorization-checked against the caller -
* ad-hoc runs are never persisted and so never hit {@link #validate(Policy)}, and the worker
* thread that later delivers has no principal, so this is their only access gate.
*
* @throws IllegalArgumentException if the type is unknown or the config is invalid/inaccessible
*/
public void validateOutput(OutputSpec output) {
outputSinkFor(output).validate(output);
}
private PolicyTrigger triggerFor(TriggerConfig config) {
@@ -18,6 +18,7 @@ import stirling.software.proprietary.policy.model.InputSpec;
import stirling.software.proprietary.policy.model.PolicyInputs;
import stirling.software.proprietary.policy.s3.S3Config;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3ConnectionResolver;
import stirling.software.proprietary.policy.s3.S3Identities;
import software.amazon.awssdk.core.exception.SdkException;
@@ -36,15 +37,14 @@ import software.amazon.awssdk.services.s3.model.S3Object;
* Reads input files from an Amazon S3 (or S3-compatible) bucket; each listed object is its own unit
* of work, claimed through the {@link ResolveContext} ledger and tracked in place. Identity and
* version gate come from {@link S3Identities}, so the steady-state sweep never downloads content.
* Options (see {@link S3Config}): "bucket" (required), "region" (default us-east-1), "prefix" (only
* keys starting with it are read), "endpoint" (S3-compatible stores such as MinIO; path-style
* addressing is used automatically), "accessKeyId" and "secretAccessKey" (required; requests are
* never signed with the server's own AWS identity), and "mode" which is "consume" (default: a
* processed object is deleted once every policy that claimed it has settled successfully and it is
* still the version that ran; failures stay in place and are not retried until they change) or
* "snapshot" (stateless, every run sees the full set). Keys ending in "/" (folder placeholders) and
* keys with a dot-prefixed path segment are never picked up, mirroring the folder source's
* hidden-file rule.
* Options: "connectionId" references the stored S3 connection (an {@code IntegrationConfig} owning
* bucket, region, endpoint, and credentials - resolved by {@link S3ConnectionResolver}); "prefix"
* (only keys starting with it are read) and "mode" are per-source, where mode is "consume"
* (default: a processed object is deleted once every policy that claimed it has settled
* successfully and it is still the version that ran; failures stay in place and are not retried
* until they change) or "snapshot" (stateless, every run sees the full set). Keys ending in "/"
* (folder placeholders) and keys with a dot-prefixed path segment are never picked up, mirroring
* the folder source's hidden-file rule.
*/
@Slf4j
@Service
@@ -55,6 +55,7 @@ public class S3InputSource implements InputSource {
private static final String TYPE = "s3";
private final S3ConnectionPool connectionPool;
private final S3ConnectionResolver connectionResolver;
@Override
public String type() {
@@ -67,12 +68,12 @@ public class S3InputSource implements InputSource {
}
/**
* Fails fast at save time: bad config shape, a private endpoint without the operator opt-in, or
* a bucket the supplied credentials cannot list.
* Fails fast at save time: an unknown/disabled/unusable connection, bad config shape, a private
* endpoint without the operator opt-in, or a bucket the connection cannot list.
*/
@Override
public void validate(InputSpec spec) {
S3Config config = S3Config.from(spec.options());
S3Config config = connectionResolver.resolve(spec.options());
try {
connectionPool.clientFor(config).listObjectsV2(listRequest(config).maxKeys(1).build());
} catch (SdkException e) {
@@ -89,7 +90,7 @@ public class S3InputSource implements InputSource {
@Override
public List<ResolvedInput> resolve(InputSpec spec, ResolveContext ctx) throws IOException {
S3Config config = S3Config.from(spec.options());
S3Config config = connectionResolver.resolve(spec.options());
S3Client client = connectionPool.clientFor(config);
// A listing failure propagates so the sweep reads it as "could not list" (which vetoes
// presence cleanup), never as "verifiably no objects".
@@ -27,6 +27,7 @@ import stirling.software.proprietary.policy.ledger.ProcessedLedger;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.s3.S3Config;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3ConnectionResolver;
import stirling.software.proprietary.policy.s3.S3Identities;
import software.amazon.awssdk.core.exception.SdkException;
@@ -59,6 +60,7 @@ public class S3OutputSink implements PolicyOutputSink {
private static final String TYPE = "s3";
private final S3ConnectionPool connectionPool;
private final S3ConnectionResolver connectionResolver;
private final ProcessedLedger processedLedger;
@Override
@@ -72,19 +74,19 @@ public class S3OutputSink implements PolicyOutputSink {
}
/**
* Config shape and endpoint guard only - no network probe, since write-only credentials
* (s3:PutObject without s3:ListBucket) are a legitimate setup for an output bucket and a
* listing probe would wrongly reject them.
* Connection resolution (including the saving user's right to use it) and endpoint guard only -
* no network probe, since write-only credentials (s3:PutObject without s3:ListBucket) are a
* legitimate setup for an output bucket and a listing probe would wrongly reject them.
*/
@Override
public void validate(OutputSpec spec) {
connectionPool.clientFor(S3Config.from(spec.options()));
connectionPool.clientFor(connectionResolver.resolve(spec.options()));
}
@Override
public List<ResultFile> deliver(
OutputDelivery delivery, List<Resource> outputs, OutputSpec spec) throws IOException {
S3Config config = S3Config.from(spec.options());
S3Config config = connectionResolver.resolve(spec.options());
S3Client client = connectionPool.clientFor(config);
List<ResultFile> results = new ArrayList<>();
@@ -0,0 +1,212 @@
package stirling.software.proprietary.policy.s3;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceStore;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.security.repository.TeamRepository;
import tools.jackson.databind.ObjectMapper;
/**
* One-time, idempotent extraction of legacy embedded S3 credentials into stored connections:
* sources and policy outputs written before connections shipped carry bucket/credentials in their
* own options; this rewrites each to reference a (deduplicated) S3 {@link IntegrationConfig} and
* keeps only per-use options (prefix, mode). MUST be programmatic - the option JSON is encrypted at
* the application layer, so no SQL migration can read it.
*
* <p>Idempotent by construction: rewritten rows no longer embed credentials, so re-runs find
* nothing to do. Connections are deduplicated against both this run's extractions and existing S3
* connections; a concurrent multi-node boot can at worst create a redundant connection row, never
* corrupt a source. Ownership follows the owning row: team-scoped when the source/policy has a
* team, server-scoped otherwise (single-operator self-hosted).
*/
@Slf4j
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class EmbeddedS3CredentialMigration {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private static final List<String> CONNECTION_OPTIONS =
List.of("bucket", "region", "endpoint", "accessKeyId", "secretAccessKey");
// Field separator for the dedup key: a unit-separator control char that cannot appear in a
// bucket/region/endpoint/credential, so distinct field sets can never collide.
private static final char DELIMITER = '\u001f';
private final SourceStore sourceStore;
private final PolicyStore policyStore;
private final IntegrationConfigRepository connections;
private final TeamRepository teamRepository;
@EventListener(ApplicationReadyEvent.class)
@Transactional
public void migrate() {
Map<String, IntegrationConfig> byCredentialKey = indexExistingConnections();
int migrated = 0;
for (Source source : sourceStore.all()) {
if (!"s3".equals(source.type()) || !embedsCredentials(source.options())) {
continue;
}
IntegrationConfig connection =
connectionFor(source.options(), source.teamId(), byCredentialKey);
sourceStore.save(withOptions(source, referencing(connection, source.options(), true)));
migrated++;
}
for (Policy policy : policyStore.all()) {
OutputSpec output = policy.output();
if (!"s3".equals(output.type()) || !embedsCredentials(output.options())) {
continue;
}
IntegrationConfig connection =
connectionFor(output.options(), policy.teamId(), byCredentialKey);
policyStore.save(
withOutput(
policy,
new OutputSpec(
output.type(),
referencing(connection, output.options(), false))));
migrated++;
}
if (migrated > 0) {
log.info("Extracted embedded S3 credentials from {} row(s) into connections", migrated);
}
}
private static boolean embedsCredentials(Map<String, Object> options) {
return options.get("accessKeyId") != null;
}
/** Reuses an existing connection with identical coordinates+credentials, else creates one. */
private IntegrationConfig connectionFor(
Map<String, Object> options, Long teamId, Map<String, IntegrationConfig> byKey) {
String key = credentialKey(options);
IntegrationConfig existing = byKey.get(key);
if (existing != null) {
return existing;
}
IntegrationConfig connection = new IntegrationConfig();
connection.setIntegrationType(IntegrationType.S3);
connection.setName(connectionName(options, byKey));
connection.setEnabled(true);
connection.setLocked(false);
connection.setDefaultAccess(DefaultAccessPolicy.EXPLICIT_ONLY);
Team team = teamId == null ? null : teamRepository.findById(teamId).orElse(null);
if (team != null) {
connection.setScope(OwnerScope.TEAM);
connection.setOwnerTeam(team);
} else {
// No team (teamless self-hosted, or a source whose team was since deleted): server
// scope, i.e. admin-owned. An orphaned-team source's non-admin editor would then need
// an admin to re-share the connection - acceptable for the narrow orphaned case.
connection.setScope(OwnerScope.SERVER);
}
Map<String, Object> config = new LinkedHashMap<>();
for (String option : CONNECTION_OPTIONS) {
Object value = options.get(option);
if (value != null && !value.toString().isBlank()) {
config.put(option, value);
}
}
connection.setConfig(OBJECT_MAPPER.writeValueAsString(config));
IntegrationConfig saved = connections.save(connection);
byKey.put(key, saved);
return saved;
}
/** The rewritten options: the connection reference plus per-use settings only. */
private static Map<String, Object> referencing(
IntegrationConfig connection, Map<String, Object> legacy, boolean keepMode) {
Map<String, Object> options = new LinkedHashMap<>();
options.put(S3ConnectionResolver.CONNECTION_ID_OPTION, connection.getId());
Object prefix = legacy.get("prefix");
if (prefix != null && !prefix.toString().isBlank()) {
options.put("prefix", prefix);
}
Object mode = legacy.get("mode");
if (keepMode && mode != null && !mode.toString().isBlank()) {
options.put("mode", mode);
}
return options;
}
private Map<String, IntegrationConfig> indexExistingConnections() {
Map<String, IntegrationConfig> byKey = new LinkedHashMap<>();
for (IntegrationConfig connection : connections.findAll()) {
if (connection.getIntegrationType() != IntegrationType.S3) {
continue;
}
try {
Map<String, Object> config =
OBJECT_MAPPER.readValue(connection.getConfig(), Map.class);
byKey.putIfAbsent(credentialKey(config), connection);
} catch (Exception e) {
log.debug(
"Skipping unreadable S3 connection {} while indexing: {}",
connection.getId(),
e.getMessage());
}
}
return byKey;
}
private static String credentialKey(Map<String, Object> options) {
StringBuilder key = new StringBuilder();
for (String option : CONNECTION_OPTIONS) {
Object value = options.get(option);
key.append(value == null ? "" : value.toString().trim()).append(DELIMITER);
}
return key.toString();
}
private static String connectionName(
Map<String, Object> options, Map<String, IntegrationConfig> byKey) {
String base = "S3: " + options.getOrDefault("bucket", "bucket");
long sameName = byKey.values().stream().filter(c -> c.getName().startsWith(base)).count();
return sameName == 0 ? base : base + " (" + (sameName + 1) + ")";
}
private static Source withOptions(Source source, Map<String, Object> options) {
return new Source(
source.id(),
source.name(),
source.type(),
options,
source.enabled(),
source.owner(),
source.teamId());
}
private static Policy withOutput(Policy policy, OutputSpec output) {
return new Policy(
policy.id(),
policy.name(),
policy.owner(),
policy.enabled(),
policy.trigger(),
policy.sourceIds(),
policy.steps(),
output,
policy.teamId());
}
}
@@ -0,0 +1,56 @@
package stirling.software.proprietary.policy.s3;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.integration.service.IntegrationConfigUsageCheck;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.source.SourceStore;
import stirling.software.proprietary.policy.store.PolicyStore;
/**
* Reports the policy sources and pipeline outputs referencing an S3 connection, so the connection
* cannot be deleted out from under them (mirrors {@code SourceController}'s referenced-source
* delete guard). Scans in memory - fine at admin-dashboard scale, always consistent with the live
* stores.
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyS3ConnectionUsageCheck implements IntegrationConfigUsageCheck {
private final SourceStore sourceStore;
private final PolicyStore policyStore;
@Override
public List<String> usagesOf(long configId) {
List<String> usages = new ArrayList<>();
for (Source source : sourceStore.all()) {
if (references(source.options(), configId)) {
usages.add("source '" + source.name() + "'");
}
}
for (Policy policy : policyStore.all()) {
if (references(policy.output().options(), configId)) {
usages.add("pipeline '" + policy.name() + "'");
}
}
return usages;
}
private static boolean references(Map<String, Object> options, long configId) {
try {
Long reference = S3ConnectionResolver.connectionId(options);
return reference != null && reference == configId;
} catch (IllegalArgumentException unparseable) {
return false;
}
}
}
@@ -5,10 +5,12 @@ import java.net.URISyntaxException;
import java.util.Map;
/**
* Connection settings shared by the S3 input source and output sink, parsed from a spec's options
* map. Credentials are required: there is deliberately no fallback to the server's own AWS
* credential chain, so user-supplied config can never borrow the host's identity. {@code snapshot}
* is input-only and ignored by the sink.
* The fully resolved connection settings the S3 input source and output sink run with - normally
* produced by {@link S3ConnectionResolver} merging a stored connection (bucket, region, endpoint,
* credentials) with per-use options (prefix, mode), or parsed directly from legacy options that
* still embed credentials. Credentials are required: there is deliberately no fallback to the
* server's own AWS credential chain, so user-supplied config can never borrow the host's identity.
* {@code snapshot} is input-only and ignored by the sink.
*/
public record S3Config(
String bucket,
@@ -0,0 +1,151 @@
package stirling.software.proprietary.policy.s3;
import java.util.LinkedHashMap;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.ObjectMapper;
/**
* Turns a source's or output's options into a full {@link S3Config} by dereferencing its {@code
* connectionId} to a stored S3 {@link IntegrationConfig} (the connection owns bucket, region,
* endpoint, and credentials; the options own per-use settings such as prefix and mode). Options
* with no {@code connectionId} fall back to legacy embedded credentials, so rows written before
* connections shipped keep working until {@link EmbeddedS3CredentialMigration} rewrites them.
*
* <p>When an authenticated caller is present (save-time validation), they must be allowed to use
* the connection. Background sweeps and deliveries run with no caller and skip that check: the
* referencing source or policy was access-checked when it was saved.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3ConnectionResolver {
static final String CONNECTION_ID_OPTION = "connectionId";
private static final String PREFIX_OPTION = "prefix";
private static final String MODE_OPTION = "mode";
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private final IntegrationConfigRepository connections;
private final OwnershipService ownership;
private final UserService userService;
public S3Config resolve(Map<String, Object> options) {
Long connectionId = connectionId(options);
if (connectionId == null) {
// Legacy embedded credentials, pending migration.
return S3Config.from(options);
}
IntegrationConfig connection =
connections
.findById(connectionId)
.filter(cfg -> cfg.getIntegrationType() == IntegrationType.S3)
.filter(this::usableByCurrentUser)
// Existence and access collapse into one error: a caller must not be able
// to tell "no such connection" from "someone else's connection" and
// enumerate ids. The id/name are never echoed.
.orElseThrow(
() ->
new IllegalArgumentException(
"unknown or inaccessible s3 connection"));
if (!connection.isEnabled()) {
throw new IllegalArgumentException("s3 connection is disabled");
}
Map<String, Object> merged = new LinkedHashMap<>(connectionConfig(connection));
copyPerUseOption(options, merged, PREFIX_OPTION);
copyPerUseOption(options, merged, MODE_OPTION);
return S3Config.from(merged);
}
/** The {@code connectionId} option as a long, or null when the options are legacy-embedded. */
static Long connectionId(Map<String, Object> options) {
Object reference = options.get(CONNECTION_ID_OPTION);
if (reference == null || (reference instanceof String s && s.isBlank())) {
return null;
}
if (reference instanceof Number number) {
return number.longValue();
}
try {
return Long.valueOf(reference.toString().trim());
} catch (NumberFormatException e) {
throw new IllegalArgumentException(
"s3 'connectionId' is not a valid connection reference: " + reference);
}
}
/**
* Whether the current caller may use this connection. With no principal - a background sweep or
* delivery on a worker thread that carries no {@code SecurityContext} - access is treated as
* already established: stored policies are validated with the caller present at save time, and
* ad-hoc runs are validated on the request thread before dispatch (see {@code
* PolicyValidator#validateOutput}). A missing principal must therefore never be the ONLY thing
* standing between a caller and a connection, or the check becomes a confused deputy.
*/
private boolean usableByCurrentUser(IntegrationConfig connection) {
User user = currentUser();
return user == null || ownership.canUse(ResourceType.INTEGRATION_CONFIG, connection, user);
}
// Mirrors ResourceAccessSecurity's principal resolution; null when unauthenticated.
private User currentUser() {
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
if (auth == null || !auth.isAuthenticated()) {
return null;
}
Object principal = auth.getPrincipal();
if (principal instanceof User user) {
return user;
}
if (principal instanceof UserDetails userDetails) {
return userService.findByUsername(userDetails.getUsername()).orElse(null);
}
if (principal instanceof String username && !"anonymousUser".equals(username)) {
return userService.findByUsername(username).orElse(null);
}
return null;
}
private static Map<String, Object> connectionConfig(IntegrationConfig connection) {
String json = connection.getConfig();
if (json == null || json.isBlank()) {
return Map.of();
}
try {
return OBJECT_MAPPER.readValue(
json, new TypeReference<LinkedHashMap<String, Object>>() {});
} catch (Exception e) {
throw new IllegalArgumentException(
"s3 connection '" + connection.getName() + "' has unreadable config", e);
}
}
private static void copyPerUseOption(
Map<String, Object> options, Map<String, Object> merged, String key) {
Object value = options.get(key);
if (value != null && !value.toString().isBlank()) {
merged.put(key, value);
}
}
}
@@ -0,0 +1,51 @@
package stirling.software.proprietary.policy.s3;
import java.net.URI;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.cluster.s3.S3Clients;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.service.IntegrationConfigValidator;
/**
* The S3 connection schema, enforced when an S3 {@link IntegrationType} config is saved: bucket and
* credentials required, endpoint an http(s) URL that must not reach private addresses without the
* operator opt-in - the same rules {@link S3ConnectionPool} enforces before signing, moved to save
* time so a bad connection fails in the form rather than in a sweep.
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3IntegrationValidator implements IntegrationConfigValidator {
private final ApplicationProperties applicationProperties;
@Override
public IntegrationType type() {
return IntegrationType.S3;
}
@Override
public void validate(Map<String, Object> config) {
S3Config parsed = S3Config.from(config);
if (parsed.endpoint() == null) {
return;
}
try {
S3Clients.validateEndpointHost(
URI.create(parsed.endpoint()),
applicationProperties.getPolicies().isAllowPrivateS3Endpoints(),
"S3 connection endpoint",
"set policies.allowPrivateS3Endpoints=true to opt in (e.g. for a local"
+ " MinIO).");
} catch (IllegalStateException e) {
throw new IllegalArgumentException(e.getMessage(), e);
}
}
}
@@ -103,7 +103,6 @@ public class User implements UserDetails, Serializable {
@ElementCollection
@MapKeyColumn(name = "setting_key")
@Lob
@Column(name = "setting_value", columnDefinition = "text")
@CollectionTable(name = "user_settings", joinColumns = @JoinColumn(name = "user_id"))
@JsonIgnore
@@ -13,9 +13,9 @@ import java.util.Map;
import java.util.Optional;
import java.util.Set;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
@@ -52,7 +52,58 @@ class IntegrationConfigServiceTest {
@Mock
private stirling.software.proprietary.access.repository.ResourceGrantRepository grantRepository;
@InjectMocks private IntegrationConfigService service;
@Mock private IntegrationConfigValidator validator;
@Mock private IntegrationConfigUsageCheck usageCheck;
private IntegrationConfigService service;
@BeforeEach
void setUp() {
service =
new IntegrationConfigService(
repository,
ownership,
secretMasker,
grantRepository,
List.of(validator),
List.of(usageCheck));
}
@Test
void createRejectsAConfigItsTypeValidatorRefuses() {
when(secretMasker.sanitize(any())).thenReturn(Map.of());
when(validator.type()).thenReturn(IntegrationType.API);
org.mockito.Mockito.doThrow(new IllegalArgumentException("api config needs a 'url'"))
.when(validator)
.validate(any());
assertThatThrownBy(
() ->
service.create(
request(IntegrationType.API, OwnerScope.USER, null),
user(7)))
.isInstanceOf(ResponseStatusException.class)
.satisfies(
e ->
assertThat(((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.BAD_REQUEST));
}
@Test
void deleteRefusedWhileAnythingStillReferencesTheConfig() {
IntegrationConfig cfg = config(9L);
when(repository.findById(9L)).thenReturn(Optional.of(cfg));
when(ownership.canManage(any(), eq(cfg), any())).thenReturn(true);
when(usageCheck.usagesOf(9L)).thenReturn(List.of("source 'Claims intake'"));
assertThatThrownBy(() -> service.delete(9L, user(7)))
.isInstanceOf(ResponseStatusException.class)
.satisfies(
e ->
assertThat(((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.CONFLICT));
verify(repository, org.mockito.Mockito.never()).delete(any(IntegrationConfig.class));
}
@Test
void createDelegatesOwnershipAndSanitizesConfig() {
@@ -4,6 +4,7 @@ import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -194,6 +195,29 @@ class PolicyControllerTest {
assertThat(((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.BAD_REQUEST));
}
@Test
@DisplayName("rejects an ad-hoc output the caller cannot use, on the request thread")
void rejectsUnauthorizedAdHocOutput() {
// The confused-deputy guard: an S3 output referencing a connection the caller may not
// use is validated here (principal present) and refused before any worker dispatch.
PipelineDefinition definition =
new PipelineDefinition(
"pipe",
List.of(new PipelineStep("/api/v1/misc/compress-pdf", null)),
new OutputSpec("s3", Map.of("connectionId", 999)));
doThrow(new IllegalArgumentException("unknown or inaccessible s3 connection"))
.when(policyValidator)
.validateOutput(any());
assertThatThrownBy(() -> controller.run(definition, new PolicyRunFiles()))
.isInstanceOf(ResponseStatusException.class)
.satisfies(
e ->
assertThat(((ResponseStatusException) e).getStatusCode())
.isEqualTo(HttpStatus.BAD_REQUEST));
verify(policyRunner, never()).runAdHoc(any(), any(), any());
}
}
@Nested
@@ -82,6 +82,28 @@ class PolicyValidatorTest {
assertTrue(ex.getMessage().contains("schedule"));
}
@Test
void validateOutputDelegatesToTheSink() {
when(outputSink.supports(any())).thenReturn(true);
OutputSpec output = new OutputSpec("s3", Map.of("connectionId", 1));
validator.validateOutput(output);
verify(outputSink).validate(output);
}
@Test
void validateOutputSurfacesAnInaccessibleConnection() {
when(outputSink.supports(any())).thenReturn(true);
doThrow(new IllegalArgumentException("unknown or inaccessible s3 connection"))
.when(outputSink)
.validate(any());
assertThrows(
IllegalArgumentException.class,
() -> validator.validateOutput(new OutputSpec("s3", Map.of("connectionId", 1))));
}
@Test
void rejectsAnUnknownTriggerType() {
when(trigger.type()).thenReturn("schedule");
@@ -23,6 +23,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.policy.ledger.InProcessProcessedLedger;
import stirling.software.proprietary.policy.model.InputSpec;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3TestConnections;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
@@ -82,7 +83,9 @@ class S3InputSourceMinioTest {
// The MinIO endpoint resolves to loopback, so the operator opt-in must be on.
ApplicationProperties properties = new ApplicationProperties();
properties.getPolicies().setAllowPrivateS3Endpoints(true);
source = new S3InputSource(new S3ConnectionPool(properties));
source =
new S3InputSource(
new S3ConnectionPool(properties), S3TestConnections.legacyResolver());
ledger = new InProcessProcessedLedger();
ctx = new RecordingContext();
}
@@ -161,7 +164,9 @@ class S3InputSourceMinioTest {
@Test
void aPrivateEndpointIsRejectedWithoutTheOperatorOptIn() {
S3InputSource guarded =
new S3InputSource(new S3ConnectionPool(new ApplicationProperties()));
new S3InputSource(
new S3ConnectionPool(new ApplicationProperties()),
S3TestConnections.legacyResolver());
assertThatThrownBy(() -> guarded.validate(spec(Map.of())))
.isInstanceOf(IllegalArgumentException.class)
@@ -29,6 +29,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.policy.ledger.InProcessProcessedLedger;
import stirling.software.proprietary.policy.model.InputSpec;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3TestConnections;
import software.amazon.awssdk.core.ResponseInputStream;
import software.amazon.awssdk.core.exception.SdkClientException;
@@ -64,7 +65,8 @@ class S3InputSourceTest {
void setUp() {
source =
new S3InputSource(
new S3ConnectionPool(new ApplicationProperties(), config -> s3Client));
new S3ConnectionPool(new ApplicationProperties(), config -> s3Client),
S3TestConnections.legacyResolver());
ledger = new InProcessProcessedLedger();
ctx = new RecordingContext();
}
@@ -28,6 +28,7 @@ import stirling.software.proprietary.policy.ledger.InProcessProcessedLedger;
import stirling.software.proprietary.policy.model.InputSpec;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3TestConnections;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
@@ -90,8 +91,8 @@ class S3OutputSinkMinioTest {
properties.getPolicies().setAllowPrivateS3Endpoints(true);
S3ConnectionPool pool = new S3ConnectionPool(properties);
ledger = new InProcessProcessedLedger();
sink = new S3OutputSink(pool, ledger);
source = new S3InputSource(pool);
sink = new S3OutputSink(pool, S3TestConnections.legacyResolver(), ledger);
source = new S3InputSource(pool, S3TestConnections.legacyResolver());
}
@Test
@@ -32,6 +32,7 @@ import stirling.software.proprietary.policy.ledger.InProcessProcessedLedger;
import stirling.software.proprietary.policy.ledger.ProcessedFileStatus;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.s3.S3ConnectionPool;
import stirling.software.proprietary.policy.s3.S3TestConnections;
import software.amazon.awssdk.awscore.exception.AwsServiceException;
import software.amazon.awssdk.core.exception.SdkClientException;
@@ -66,6 +67,7 @@ class S3OutputSinkTest {
sink =
new S3OutputSink(
new S3ConnectionPool(new ApplicationProperties(), config -> s3Client),
S3TestConnections.legacyResolver(),
ledger);
}
@@ -0,0 +1,212 @@
package stirling.software.proprietary.policy.s3;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicLong;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.proprietary.access.model.OwnerScope;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.source.InProcessSourceStore;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.store.InProcessPolicyStore;
import stirling.software.proprietary.security.repository.TeamRepository;
/**
* Tests for {@link EmbeddedS3CredentialMigration}: legacy embedded credentials become deduplicated
* team-scoped connections, rewritten rows keep only per-use options, and re-runs are no-ops.
*/
@ExtendWith(MockitoExtension.class)
class EmbeddedS3CredentialMigrationTest {
@Mock private IntegrationConfigRepository connections;
@Mock private TeamRepository teamRepository;
private final InProcessSourceStore sourceStore = new InProcessSourceStore();
private final InProcessPolicyStore policyStore = new InProcessPolicyStore();
private EmbeddedS3CredentialMigration migration;
@BeforeEach
void setUp() {
migration =
new EmbeddedS3CredentialMigration(
sourceStore, policyStore, connections, teamRepository);
AtomicLong ids = new AtomicLong(100);
// Lenient: the nothing-to-migrate cases never create a connection.
lenient().when(connections.findAll()).thenReturn(List.of());
lenient()
.when(connections.save(any()))
.thenAnswer(
invocation -> {
IntegrationConfig saved = invocation.getArgument(0);
if (saved.getId() == null) {
saved.setId(ids.incrementAndGet());
}
return saved;
});
}
@Test
void extractsSharedCredentialsIntoOneTeamScopedConnection() {
Team team = new Team();
team.setId(7L);
when(teamRepository.findById(7L)).thenReturn(Optional.of(team));
Source source =
sourceStore.save(
new Source(
null,
"Claims intake",
"s3",
Map.of(
"bucket", "inbox",
"prefix", "incoming/",
"mode", "snapshot",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh"),
true,
"alice",
7L));
Policy policy =
policyStore.save(
new Policy(
null,
"Rotate",
"alice",
true,
null,
List.of(source.id()),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
new OutputSpec(
"s3",
Map.of(
"bucket", "inbox",
"prefix", "processed/",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh")),
7L));
migration.migrate();
// Same bucket + credentials on both rows: exactly one connection extracted.
verify(connections, times(1)).save(any());
Map<String, Object> sourceOptions = sourceStore.get(source.id()).orElseThrow().options();
assertEquals(101L, sourceOptions.get("connectionId"));
assertEquals("incoming/", sourceOptions.get("prefix"));
assertEquals("snapshot", sourceOptions.get("mode"));
assertNull(sourceOptions.get("accessKeyId"));
assertNull(sourceOptions.get("secretAccessKey"));
assertNull(sourceOptions.get("bucket"));
Map<String, Object> outputOptions =
policyStore.get(policy.id()).orElseThrow().output().options();
assertEquals(101L, outputOptions.get("connectionId"));
assertEquals("processed/", outputOptions.get("prefix"));
assertNull(outputOptions.get("secretAccessKey"));
}
@Test
void connectionOwnershipFollowsTheSourceTeam() {
Team team = new Team();
team.setId(7L);
when(teamRepository.findById(7L)).thenReturn(Optional.of(team));
sourceStore.save(s3Source("teamed", 7L));
migration.migrate();
verify(connections)
.save(
org.mockito.ArgumentMatchers.argThat(
connection ->
connection.getScope() == OwnerScope.TEAM
&& connection.getOwnerTeam() == team));
}
@Test
void teamlessRowsBecomeServerScopedConnections() {
sourceStore.save(s3Source("solo", null));
migration.migrate();
verify(connections)
.save(
org.mockito.ArgumentMatchers.argThat(
connection -> connection.getScope() == OwnerScope.SERVER));
}
@Test
void aSecondRunFindsNothingToDo() {
sourceStore.save(s3Source("once", null));
migration.migrate();
migration.migrate();
// One connection from the first run; the rewritten source no longer embeds credentials.
verify(connections, times(1)).save(any());
}
@Test
void nonS3AndAlreadyMigratedRowsAreUntouched() {
Source folder =
sourceStore.save(
new Source(
null,
"Folder",
"folder",
Map.of("directory", "/in"),
true,
"alice",
null));
Source migrated =
sourceStore.save(
new Source(
null,
"Done already",
"s3",
Map.of("connectionId", 55L, "prefix", "in/"),
true,
"alice",
null));
migration.migrate();
verify(connections, times(0)).save(any());
assertEquals(
Map.of("directory", "/in"), sourceStore.get(folder.id()).orElseThrow().options());
assertEquals(
Map.of("connectionId", 55L, "prefix", "in/"),
sourceStore.get(migrated.id()).orElseThrow().options());
}
private static Source s3Source(String name, Long teamId) {
return new Source(
null,
name,
"s3",
Map.of(
"bucket", "inbox",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh"),
true,
"alice",
teamId);
}
}
@@ -0,0 +1,52 @@
package stirling.software.proprietary.policy.s3;
import static org.assertj.core.api.Assertions.assertThat;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.source.InProcessSourceStore;
import stirling.software.proprietary.policy.source.Source;
import stirling.software.proprietary.policy.store.InProcessPolicyStore;
/** Tests for {@link PolicyS3ConnectionUsageCheck}'s reference scan across sources and outputs. */
class PolicyS3ConnectionUsageCheckTest {
private final InProcessSourceStore sourceStore = new InProcessSourceStore();
private final InProcessPolicyStore policyStore = new InProcessPolicyStore();
private final PolicyS3ConnectionUsageCheck check =
new PolicyS3ConnectionUsageCheck(sourceStore, policyStore);
@Test
void reportsSourcesAndOutputsReferencingTheConnection() {
sourceStore.save(
new Source(
null,
"Claims intake",
"s3",
Map.of("connectionId", 5L, "prefix", "in/"),
true,
"alice",
null));
policyStore.save(
new Policy(
null,
"Rotate",
"alice",
true,
null,
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
new OutputSpec("s3", Map.of("connectionId", "5")),
null));
assertThat(check.usagesOf(5))
.containsExactlyInAnyOrder("source 'Claims intake'", "pipeline 'Rotate'");
assertThat(check.usagesOf(6)).isEmpty();
}
}
@@ -0,0 +1,149 @@
package stirling.software.proprietary.policy.s3;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
import java.util.Map;
import java.util.Optional;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.context.SecurityContextHolder;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
/**
* Tests for {@link S3ConnectionResolver}: connection dereferencing with per-use overrides, the
* legacy embedded fallback, and the save-time access check that background sweeps skip.
*/
@ExtendWith(MockitoExtension.class)
class S3ConnectionResolverTest {
@Mock private IntegrationConfigRepository connections;
@Mock private OwnershipService ownership;
@Mock private UserService userService;
@AfterEach
void clearSecurityContext() {
SecurityContextHolder.clearContext();
}
@Test
void resolvesAConnectionAndMergesPerUseOptions() {
when(connections.findById(9L)).thenReturn(Optional.of(s3Connection(9L, true)));
S3Config config =
resolver()
.resolve(
Map.of(
"connectionId", 9L,
"prefix", "incoming/",
"mode", "snapshot"));
assertEquals("inbox", config.bucket());
assertEquals("AKIAEXAMPLE", config.accessKeyId());
assertEquals("incoming/", config.prefix());
assertTrue(config.snapshot());
}
@Test
void acceptsAStringConnectionReference() {
when(connections.findById(9L)).thenReturn(Optional.of(s3Connection(9L, true)));
assertEquals("inbox", resolver().resolve(Map.of("connectionId", "9")).bucket());
}
@Test
void fallsBackToLegacyEmbeddedCredentials() {
S3Config config =
resolver()
.resolve(
Map.of(
"bucket", "legacy",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh"));
assertEquals("legacy", config.bucket());
}
@Test
void rejectsUnknownDisabledOrWrongTypeConnections() {
when(connections.findById(1L)).thenReturn(Optional.empty());
assertThrows(
IllegalArgumentException.class,
() -> resolver().resolve(Map.of("connectionId", 1L)));
when(connections.findById(2L)).thenReturn(Optional.of(s3Connection(2L, false)));
assertThrows(
IllegalArgumentException.class,
() -> resolver().resolve(Map.of("connectionId", 2L)));
IntegrationConfig mcp = s3Connection(3L, true);
mcp.setIntegrationType(IntegrationType.MCP);
when(connections.findById(3L)).thenReturn(Optional.of(mcp));
assertThrows(
IllegalArgumentException.class,
() -> resolver().resolve(Map.of("connectionId", 3L)));
}
@Test
void anAuthenticatedSaverMustBeAllowedToUseTheConnection() {
when(connections.findById(9L)).thenReturn(Optional.of(s3Connection(9L, true)));
User saver = new User();
saver.setUsername("alice");
SecurityContextHolder.getContext()
.setAuthentication(
new UsernamePasswordAuthenticationToken(saver, null, java.util.List.of()));
when(ownership.canUse(any(), any(IntegrationConfig.class), eq(saver))).thenReturn(false);
// Denied reads the same as unknown and never echoes the connection name, so ids can't be
// enumerated by probing.
assertThrows(
IllegalArgumentException.class,
() -> resolver().resolve(Map.of("connectionId", 9L)));
try {
resolver().resolve(Map.of("connectionId", 9L));
} catch (IllegalArgumentException e) {
org.junit.jupiter.api.Assertions.assertFalse(
e.getMessage().contains("Claims bucket"),
"access-denied error must not leak the connection name");
}
}
@Test
void backgroundSweepsWithNoUserSkipTheAccessCheck() {
when(connections.findById(9L)).thenReturn(Optional.of(s3Connection(9L, true)));
// No authentication in the context: resolution succeeds without consulting ownership.
assertEquals("inbox", resolver().resolve(Map.of("connectionId", 9L)).bucket());
}
private S3ConnectionResolver resolver() {
return new S3ConnectionResolver(connections, ownership, userService);
}
private static IntegrationConfig s3Connection(long id, boolean enabled) {
IntegrationConfig connection = new IntegrationConfig();
connection.setId(id);
connection.setIntegrationType(IntegrationType.S3);
connection.setName("Claims bucket");
connection.setEnabled(enabled);
connection.setConfig(
"{\"bucket\":\"inbox\",\"accessKeyId\":\"AKIAEXAMPLE\","
+ "\"secretAccessKey\":\"shh\"}");
return connection;
}
}
@@ -0,0 +1,71 @@
package stirling.software.proprietary.policy.s3;
import static org.assertj.core.api.Assertions.assertThatCode;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import java.util.Map;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.integration.model.IntegrationType;
/**
* Tests for {@link S3IntegrationValidator}: the S3 connection schema fails at save time - missing
* credentials, bad endpoints, and private endpoints without the operator opt-in.
*/
class S3IntegrationValidatorTest {
@Test
void acceptsACompleteConnection() {
assertThatCode(
() ->
validator(false)
.validate(
Map.of(
"bucket", "inbox",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh")))
.doesNotThrowAnyException();
}
@Test
void rejectsMissingCredentialsOrBucket() {
assertThatThrownBy(() -> validator(false).validate(Map.of("bucket", "inbox")))
.isInstanceOf(IllegalArgumentException.class);
assertThatThrownBy(
() ->
validator(false)
.validate(
Map.of(
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh")))
.isInstanceOf(IllegalArgumentException.class);
}
@Test
void rejectsAPrivateEndpointWithoutTheOperatorOptIn() {
Map<String, Object> config =
Map.of(
"bucket", "inbox",
"accessKeyId", "AKIAEXAMPLE",
"secretAccessKey", "shh",
"endpoint", "http://localhost:9000");
assertThatThrownBy(() -> validator(false).validate(config))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("allowPrivateS3Endpoints");
assertThatCode(() -> validator(true).validate(config)).doesNotThrowAnyException();
}
@Test
void itOnlyClaimsTheS3Type() {
org.junit.jupiter.api.Assertions.assertEquals(IntegrationType.S3, validator(false).type());
}
private static S3IntegrationValidator validator(boolean allowPrivateEndpoints) {
ApplicationProperties properties = new ApplicationProperties();
properties.getPolicies().setAllowPrivateS3Endpoints(allowPrivateEndpoints);
return new S3IntegrationValidator(properties);
}
}
@@ -0,0 +1,24 @@
package stirling.software.proprietary.policy.s3;
import static org.mockito.Mockito.mock;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.service.UserService;
/** Test fixtures for S3 connection plumbing shared across the policy S3 tests. */
public final class S3TestConnections {
private S3TestConnections() {}
/**
* A resolver for tests whose options embed credentials directly (the legacy pass-through path),
* so its collaborators are never touched.
*/
public static S3ConnectionResolver legacyResolver() {
return new S3ConnectionResolver(
mock(IntegrationConfigRepository.class),
mock(OwnershipService.class),
mock(UserService.class));
}
}
+1 -1
View File
@@ -91,7 +91,7 @@ springBoot {
allprojects {
group = 'stirling.software'
version = '2.14.1'
version = '2.14.2'
configurations.configureEach {
exclude group: "org.springframework.boot", module: "spring-boot-starter-tomcat"
+17 -1
View File
@@ -43,14 +43,30 @@ function parseEnTranslation(): Record<string, unknown> {
}
}
const enTranslationResources = parseEnTranslation();
if (!i18next.isInitialized) {
// initImmediate: false → initialise synchronously from the inline resources
// (there's no async backend here), so i18next is ready before the first story
// renders. Without it the first render can beat init and stick on raw keys.
void i18next.use(initReactI18next).init({
lng: "en",
fallbackLng: "en",
resources: { en: { translation: parseEnTranslation() } },
resources: { en: { translation: enTranslationResources } },
interpolation: { escapeValue: false },
react: { useSuspense: false },
initImmediate: false,
});
} else {
// Something initialised i18next first (e.g. the app's async TOML backend):
// inject the shipped English copy synchronously so t() never renders raw keys.
i18next.addResourceBundle(
"en",
"translation",
enTranslationResources,
true,
true,
);
if (i18next.language !== "en") void i18next.changeLanguage("en");
}
// Start MSW once. Storybook runs in a browser so this uses the service worker.
@@ -6240,6 +6240,10 @@ muted = "muted"
nameLabel = "Name"
namePlaceholder = "e.g. Compliance escalation"
[portal.agentBuilder.status]
draft = "Draft"
published = "Published"
[portal.agentBuilder.tabs]
evals = "Evals"
scenarios = "Scenarios"
@@ -6463,7 +6467,7 @@ install = "Install"
usage = "Usage"
[portal.catalogue.detail.locked]
description = "{{name}} is included from the {{tier}} plan. Upgrade to embed it."
description = "{{name}} is included from the {{plan}}. Upgrade to embed it."
title = "Not available on your plan"
[portal.catalogue.detail.preview]
@@ -6530,6 +6534,35 @@ title = "No components available"
description = "GA components are available on Pay-as-you-go; a few Beta components are enterprise-only. Locked cards show an upgrade nudge."
title = "Some components need a paid plan"
[portal.connections]
createTitle = "New S3 connection"
delete = "Delete"
edit = "Edit"
editTitle = "Edit S3 connection"
subtitle = "Reusable S3 credentials that sources and pipeline outputs connect to."
[portal.connections.actions]
new = "New connection"
[portal.connections.empty]
description = "Add an S3 connection to reuse the same bucket and credentials across sources and pipeline outputs."
title = "No connections yet"
[portal.connections.picker]
cancel = "Cancel"
createNew = "New connection..."
placeholder = "Select a connection"
save = "Save connection"
[portal.connections.s3.fields]
name = "Connection name"
namePlaceholder = "e.g. Claims bucket"
[portal.connections.table]
bucket = "Bucket"
name = "Name"
region = "Region"
[portal.docs.authentication]
codeCaption = "every request"
eyebrow = "GETTING STARTED"
@@ -7341,10 +7374,6 @@ operations_one = "Operation ({{count}})"
operations_other = "Operations ({{count}})"
output = "Output"
removeStep = "Remove operation"
s3Configure = "Configure"
s3Done = "Done"
s3ModalTitle = "Amazon S3 output"
s3NotConfigured = "Not configured"
s3PrefixHelp = "Outputs are uploaded under this key prefix."
save = "Save changes"
scheduleEvery = "Run every"
@@ -7415,6 +7444,7 @@ manual = "Manual"
schedule = "Scheduled"
[portal.policies]
defaultName = "{{category}} Policy"
subtitle = "Standing automations that enforce a tool pipeline on every document. Each policy fires on upload or export, runs its tool chain, and saves the enforced version alongside the original."
title = "Policies"
@@ -7678,6 +7708,48 @@ soon = "Soon"
managePlan = "Manage plan"
volumeSuffix = "PDFs processed · last 30 days"
[portal.processorFlow]
footnote = "Counts are over the last 24 hours. Flow speed is illustrative."
liveBadge = "Live"
stats = "{{connected}} connected · {{processed}} PDFs processed"
title = "PDF Processor"
[portal.processorFlow.lens]
ariaLabel = "View mode"
flow = "Flow"
sankey = "Sankey"
[portal.processorFlow.outcomes]
count = "{{n}} · 24h"
failed = "Failed"
heading = "Outcomes"
success = "Delivered"
[portal.processorFlow.policies]
activeCount = "{{n}} active"
count = "{{n}} · 24h"
heading = "Policies"
setUp = "Set up"
soon = "Soon"
[portal.processorFlow.sankey]
waist = "Policies · {{n}} active"
[portal.processorFlow.sankey.empty]
description = "Connect a source and switch on a policy to watch documents flow through the processor."
title = "No flow yet"
[portal.processorFlow.sources]
comingSoonTag = "Connect"
editor = "Stirling PDF Editor"
heading = "Sources"
perDay = "{{n}} / 24h"
[portal.processorFlow.sources.comingSoon]
apiMcp = "API · MCP"
cloud = "Drive · Box · S3"
email = "Email intake"
[portal.procurement]
subtitle = "Get your team evaluated, contracted, and onboarded. Every document in one place."
title = "Procurement"
@@ -7986,34 +8058,29 @@ primaryNav = "Primary navigation"
switchApp = "Switch app"
[portal.sources]
subtitle = "Reusable input connections that feed documents into Stirling. Configure a connection once, then reference it from any number of policies. Click a row for its config and which policies use it."
subtitle = "Reusable input connections that feed documents into Stirling. Configure a source once, then reference it from any number of pipelines."
title = "Sources"
[portal.sources.actions]
agentBuilder = "Agent Builder"
connectSource = "Connect source"
[portal.sources.builder]
back = "Back to sources"
cancel = "Cancel"
create = "Create source"
createTitle = "Connect a source"
delete = "Delete"
editTitle = "Edit source"
enabled = "Enabled"
save = "Save changes"
[portal.sources.delete]
body = "Delete \"{{name}}\"? This can't be undone. Policies that reference it would need to be updated."
cancel = "Cancel"
confirm = "Delete"
title = "Delete source?"
[portal.sources.detail]
closeAriaLabel = "Close detail"
delete = "Delete source"
docs24h = "Last 24h"
docs30d = "Last 30 days"
docsTotal = "Total seen"
docsTrend = "Documents over the last 30 days"
documents = "Documents"
edit = "Edit"
notReferenced = "Not referenced by any policy, so it's safe to delete."
pause = "Pause"
resume = "Resume"
subtitle = "{{type}} · {{status}}"
usedBy = "Used by"
[portal.sources.empty]
description = "Connect a folder (and, soon, cloud storage) so your policies have somewhere to pull documents from."
title = "No sources connected yet"
@@ -8029,10 +8096,15 @@ disabled = "Disabled"
unused = "Unused"
[portal.sources.table]
documents = "Documents"
source = "Source"
status = "Status"
usedBy = "Policies"
[portal.sources.tabs]
connections = "Connections"
sources = "Sources"
[portal.sources.types.editor]
description = "Documents your team has processed in the editor, across policy and AI runs."
label = "Editor"
@@ -8080,6 +8152,10 @@ label = "Access key ID"
label = "Bucket"
placeholder = "my-company-inbox"
[portal.sources.types.s3.fields.connection]
helperText = "The stored connection holding the bucket and credentials. Reused by every source and pipeline output that references it."
label = "Connection"
[portal.sources.types.s3.fields.endpoint]
helperText = "Leave blank for Amazon S3. Set to use an S3-compatible service such as MinIO."
label = "Custom endpoint"
@@ -8109,21 +8185,14 @@ label = "Secret access key"
label = "Source"
[portal.sources.wizard]
back = "Back"
cancel = "Cancel"
continue = "Continue"
editTitle = "Edit source"
name = "Name"
namePlaceholder = "e.g. Claims intake"
save = "Save changes"
subtitle = "Step {{current}} of {{total}} · {{label}}"
title = "Connect a source"
type = "Type"
[portal.sources.wizard.steps]
chooseType = "Choose type"
configure = "Configure"
review = "Review & connect"
[portal.tier]
enterprise = "Enterprise plan"
free = "Editor plan"
pro = "Processor plan"
[portal.usage]
managePayment = "Manage Payment"
@@ -10139,6 +10208,19 @@ resetPw = "Reset password"
suspend = "Suspend"
unlock = "Unlock account"
[users.activity]
daysAgo = "{{count}}d ago"
hoursAgo = "{{count}}h ago"
justNow = "Just now"
minutesAgo = "{{count}}m ago"
monthsAgo_one = "{{count}} month ago"
monthsAgo_other = "{{count}} months ago"
never = "Never"
weeksAgo_one = "{{count}} week ago"
weeksAgo_other = "{{count}} weeks ago"
yearsAgo_one = "{{count}} year ago"
yearsAgo_other = "{{count}} years ago"
[users.cap]
addProcessor = "+ Processor"
approver = "Approves policy"
@@ -10209,7 +10291,9 @@ by = "Invited by {{who}}"
cancel = "Cancel"
count = "{{count}} pending"
desc = "Invited people who haven't joined yet. They hold a seat until they accept."
expires = "Expires"
expiresInDays_one = "Expires in {{count}} day"
expiresInDays_other = "Expires in {{count}} days"
expiresToday = "Expires today"
title = "Pending invitations"
[users.loadError]
+1 -1
View File
@@ -2,7 +2,7 @@
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
"productName": "Stirling PDF",
"mainBinaryName": "Stirling-PDF",
"version": "2.14.1",
"version": "2.14.2",
"identifier": "stirling.pdf.dev",
"build": {
"frontendDist": "../dist",
@@ -13,7 +13,6 @@ import {
import {
deserializeToolStep,
getExecutableTools,
serializeStepFromEndpoint,
serializeToolStep,
stepRequiresUpload,
type WorkingToolStep,
@@ -199,41 +198,6 @@ describe("serialize/deserialize round-trip", () => {
});
});
describe("serializeStepFromEndpoint", () => {
test("maps a wizard step's UI params to the backend contract, filling defaults", () => {
// The shape the policy setup wizard holds: an endpoint plus UI-shaped params
// (redact's `wordsToRedact`), with several fields left to their defaults.
const api = serializeStepFromEndpoint(
"/api/v1/security/auto-redact",
{ mode: "automatic", useRegex: true, wordsToRedact: ["ssn", "card"] },
dynamicRegistry,
);
expect(api.operation).toBe("/api/v1/security/auto-redact");
// wordsToRedact -> listOfText (the field the backend actually reads), and the
// frontend-only `mode` is dropped.
expect(api.parameters).toMatchObject({ listOfText: "ssn\ncard" });
expect(api.parameters).not.toHaveProperty("wordsToRedact");
expect(api.parameters).not.toHaveProperty("mode");
// Fields the wizard never set still get their defaults so the body is complete.
expect(api.parameters).toHaveProperty("wholeWordSearch");
expect(api.parameters).toHaveProperty("customPadding");
});
test("passes an unmapped endpoint's params through unchanged", () => {
expect(
serializeStepFromEndpoint(
"/api/v1/unknown/thing",
{ keep: true },
dynamicRegistry,
),
).toEqual({
operation: "/api/v1/unknown/thing",
parameters: { keep: true },
});
});
});
describe("stepRequiresUpload", () => {
const step = (params: Record<string, unknown>): WorkingToolStep => ({
toolId: "compress" as ToolId,
@@ -197,31 +197,6 @@ export function serializeToolStep(
return { operation, parameters };
}
/**
* Serialize a step held as an endpoint path plus frontend-shaped params - the form the policy setup
* wizard keeps, where params match the tool's UI shape (e.g. redact's `wordsToRedact`) rather than
* the backend contract - into the backend step contract, mapping params through the tool's
* `toApiParams` (merged over its defaults, so fields the wizard never set still get their defaults).
* The endpoint maps to a tool by path, so this works for dynamic-endpoint tools whose config
* endpoint is a function. Endpoints that map to no known tool pass through unchanged.
*/
export function serializeStepFromEndpoint(
operation: string,
params: ErasedToolParams,
registry: Partial<ToolRegistry>,
): ToolApiStep {
const match = findToolByEndpoint({ operation, parameters: params }, registry);
const config = match?.[1].operationConfig;
if (!config) return { operation, parameters: params };
const merged = { ...(config.defaultParameters ?? {}), ...params };
return {
operation: resolveEndpoint(config, merged) ?? operation,
parameters: config.toApiParams
? (config.toApiParams(merged) as Record<string, unknown>)
: {},
};
}
/**
* Find the registry tool for a stored step's endpoint: exact match for static endpoints, else
* membership in a dynamic tool's declared `endpoints` set (replaying its function can't recover a
@@ -0,0 +1,37 @@
import { describe, expect, test } from "vitest";
import { describeToolOperation } from "@app/hooks/tools/shared/toolOperationDescriptor";
interface Params {
a: number;
}
// A minimal config that type-checks against the flatten endpoint's model.
const CONFIG = {
endpoint: "/api/v1/misc/flatten" as const,
defaultParameters: { a: 1 } satisfies Params,
toApiParams: (p: Params) => ({ renderDpi: p.a }),
fromApiParams: (api: { renderDpi?: number }) => ({ a: api.renderDpi ?? 0 }),
};
describe("describeToolOperation", () => {
test("wraps the config's mappers and endpoint into a descriptor", () => {
const d = describeToolOperation("/api/v1/misc/flatten", CONFIG);
expect(d.endpoint).toBe("/api/v1/misc/flatten");
expect(d.toApi({ a: 200 })).toEqual({ renderDpi: 200 });
});
test("fromApi merges the mapped values over the defaults", () => {
const d = describeToolOperation("/api/v1/misc/flatten", CONFIG);
expect(d.fromApi({ renderDpi: 72 })).toEqual({ a: 72 });
});
test("throws when the config lacks a mapper", () => {
expect(() =>
describeToolOperation("/api/v1/misc/flatten", {
endpoint: "/api/v1/misc/flatten" as const,
defaultParameters: { a: 1 },
toApiParams: (p: Params) => ({ renderDpi: p.a }),
}),
).toThrow(/mappers/);
});
});
@@ -0,0 +1,59 @@
/**
* Typed wrapper over a tool's `toApiParams`/`fromApiParams` mappers, binding one endpoint to safe
* frontend<->backend parameter conversion.
*/
import type { ToolApiParams, ToolEndpoint } from "@app/types/toolApiTypes";
export interface ToolOperationDescriptor<E extends ToolEndpoint, TParams> {
readonly endpoint: E;
readonly defaultParameters: TParams;
toApi(params: TParams): ToolApiParams[E];
/** Backend model -> full frontend params (defaults merged under the mapped values). */
fromApi(api: ToolApiParams[E]): TParams;
}
/**
* Structural subset of a tool's config. `CE` is the config's declared endpoint type, inferred from
* the `endpoint` field: the literal for static tools, or the whole `ToolEndpoint` union for
* dynamic-endpoint tools (whose endpoint is a function typed against the union).
*/
export interface BidirectionalToolConfig<TParams, CE extends ToolEndpoint> {
endpoint: CE | null | ((params: TParams) => CE | null);
defaultParameters?: TParams;
toApiParams?(params: TParams): ToolApiParams[CE];
fromApiParams?(api: ToolApiParams[CE]): Partial<TParams>;
}
/**
* Pin a config to `endpoint` (passed explicitly, since dynamic-endpoint tools declare `endpoint` as
* a function). `E extends CE` rejects pairing a static tool's config with the wrong endpoint, while
* allowing a dynamic tool whose `CE` is the full union. Throws when mappers or defaults are missing.
*/
export function describeToolOperation<
E extends CE,
CE extends ToolEndpoint,
TParams,
>(
endpoint: E,
config: BidirectionalToolConfig<TParams, CE>,
): ToolOperationDescriptor<E, TParams> {
const { toApiParams, fromApiParams, defaultParameters } = config;
if (!toApiParams || !fromApiParams || defaultParameters === undefined) {
throw new Error(
`describeToolOperation: "${endpoint}" is missing mappers or defaults`,
);
}
return {
endpoint,
defaultParameters,
// A dynamic tool's mapper is typed against the union; narrow to this endpoint (sound - the
// runtime mapper produces this endpoint's model).
toApi: (params) => toApiParams(params) as ToolApiParams[E],
fromApi: (api) =>
({
...defaultParameters,
...fromApiParams(api as ToolApiParams[CE]),
}) as TParams,
};
}
@@ -38,7 +38,7 @@ const FREE_LICENSE_INFO: LicenseInfo = {
const BASE_NO_LOGIN_CONFIG: AppConfig = {
enableAnalytics: true,
appVersion: "2.14.1",
appVersion: "2.14.2",
serverCertificateEnabled: false,
enableAlphaFunctionality: false,
serverPort: 8080,
+28 -3
View File
@@ -270,8 +270,29 @@
--grad-banner: linear-gradient(135deg, #0f172a 0%, #111827 50%, #1a1535 100%);
}
/* Always-dark code palette. Not theme-switched. */
:root {
/* Code palette — theme-aware: a light GitHub-style box in light mode, the dark
terminal palette in dark mode. (CodeBlock renders plain text, so only the
base colours are load-bearing; the syntax slots are kept for future Shiki. */
:root,
[data-theme="light"] {
--code-bg: #f6f8fa;
--code-bg-alt: #eef1f4;
--code-bg-header: #eaeef2;
--code-text: #1f2328;
--code-dim: #656d76;
--code-muted: #8c959f;
--code-keyword: #cf222e;
--code-string: #0a3069;
--code-number: #0550ae;
--code-fn: #8250df;
--code-type: #953800;
--code-property: #0550ae;
--code-comment: #6e7781;
--code-border: #d0d7de;
/* Window-chrome traffic-light dots in the code-block header. */
--code-dot: #d0d7de;
}
[data-theme="dark"] {
--code-bg: #0f172a;
--code-bg-alt: #1e293b;
--code-bg-header: #1a2332;
@@ -286,12 +307,16 @@
--code-property: #93c5fd;
--code-comment: #475569;
--code-border: #1e293b;
/* Window-chrome traffic-light dots in the code-block header. */
--code-dot: #475569;
}
/* Radii / typography / motion / spacing / z-index — theme-stable */
:root {
/* Home hero strip navy. Theme-stable by design — the hero keeps this deep
navy in both light and dark (it's a branded surface, like the assistant
header), so it's defined once here rather than in the light/dark blocks. */
--color-hero-navy: #16213e;
--radius-xs: 0.1875rem;
--radius-sm: 0.25rem;
--radius-md: 0.375rem;
+9 -6
View File
@@ -61,11 +61,14 @@
background: var(--button-bg, transparent) !important;
}
/* Disabled buttons in dark read as a muted surface (not a dimmed accent that
still looks clickable, and not an invisible transparent pill). Covers every
variant so a disabled primary and a disabled secondary look alike. */
[data-theme="dark"] .sui-btn.mantine-Button-root:disabled:not([data-loading]),
[data-theme="dark"]
.sui-btn--primary.mantine-Button-root:disabled:not([data-loading]),
[data-theme="dark"]
.sui-btn--primary.mantine-Button-root[data-disabled]:not([data-loading]) {
background: var(--button-bg);
color: var(--button-color);
opacity: 0.55;
.sui-btn.mantine-Button-root[data-disabled]:not([data-loading]) {
background: var(--color-bg-muted);
color: var(--color-text-5);
border-color: transparent;
opacity: 1;
}
+2 -1
View File
@@ -51,7 +51,8 @@
transition: background var(--motion-fast);
}
.sui-code__copy:hover {
background: rgba(255, 255, 255, 0.05);
/* Subtle tint that reads on both the light and dark code surfaces. */
background: color-mix(in srgb, var(--code-text) 8%, transparent);
}
.sui-code__pre {
margin: 0;
+42 -31
View File
@@ -19,6 +19,12 @@ export interface TableProps<T> {
rowKey: (row: T) => string;
/** Makes rows interactive (hover + click + keyboard). */
onRowClick?: (row: T) => void;
/**
* Per-row gate for interactivity, checked only when {@link onRowClick} is set. A row for which
* this returns false is inert: no click/keyboard, and not announced as a button. Defaults to
* all rows interactive.
*/
isRowInteractive?: (row: T) => boolean;
/** Rendered in place of the body when there are no rows. */
empty?: ReactNode;
className?: string;
@@ -35,6 +41,7 @@ export function Table<T>({
rows,
rowKey,
onRowClick,
isRowInteractive,
empty,
className,
}: TableProps<T>) {
@@ -66,38 +73,42 @@ export function Table<T>({
</td>
</tr>
) : (
rows.map((row) => (
<tr
key={rowKey(row)}
className={
interactive
? "sui-table__row sui-table__row--interactive"
: "sui-table__row"
}
onClick={onRowClick ? () => onRowClick(row) : undefined}
tabIndex={interactive ? 0 : undefined}
role={interactive ? "button" : undefined}
onKeyDown={
interactive
? (e) => {
if (e.key === "Enter" || e.key === " ") {
e.preventDefault();
onRowClick?.(row);
rows.map((row) => {
const rowInteractive =
interactive && (isRowInteractive?.(row) ?? true);
return (
<tr
key={rowKey(row)}
className={
rowInteractive
? "sui-table__row sui-table__row--interactive"
: "sui-table__row"
}
onClick={rowInteractive ? () => onRowClick?.(row) : undefined}
tabIndex={rowInteractive ? 0 : undefined}
role={rowInteractive ? "button" : undefined}
onKeyDown={
rowInteractive
? (e) => {
if (e.key === "Enter" || e.key === " ") {
e.preventDefault();
onRowClick?.(row);
}
}
}
: undefined
}
>
{columns.map((c) => (
<td
key={c.key}
className={`sui-table__td sui-table__td--${c.align ?? "left"}`}
>
{c.render(row)}
</td>
))}
</tr>
))
: undefined
}
>
{columns.map((c) => (
<td
key={c.key}
className={`sui-table__td sui-table__td--${c.align ?? "left"}`}
>
{c.render(row)}
</td>
))}
</tr>
);
})
)}
</tbody>
</table>
@@ -5,6 +5,7 @@ import { Documents } from "@portal/views/Documents";
import { Pipelines } from "@portal/views/Pipelines";
import { PipelineBuilder } from "@portal/views/PipelineBuilder";
import { Sources } from "@portal/views/Sources";
import { SourceBuilder } from "@portal/views/SourceBuilder";
import { AgentBuilder } from "@portal/views/AgentBuilder";
import { Policies } from "@portal/views/Policies";
import { Components } from "@portal/views/Components";
@@ -36,6 +37,14 @@ export function ViewRouter() {
element={<PipelineBuilder />}
/>
<Route path={rel(VIEW_PATHS.sources)} element={<Sources />} />
<Route
path={`${rel(VIEW_PATHS.sources)}/new`}
element={<SourceBuilder />}
/>
<Route
path={`${rel(VIEW_PATHS.sources)}/:id`}
element={<SourceBuilder />}
/>
<Route
path={rel(VIEW_PATHS["agent-builder"])}
element={<AgentBuilder />}
+5
View File
@@ -99,6 +99,11 @@ export const AGENT_STATUS_TONE: Record<AgentStatus, "success" | "neutral"> = {
draft: "neutral",
};
export const AGENT_STATUS_LABEL: Record<AgentStatus, string> = {
published: "portal.agentBuilder.status.published",
draft: "portal.agentBuilder.status.draft",
};
/**
* Catalogue of tools an agent can be granted or denied. Surfaced as the chip
* palette in restricted mode so the deny list is picked from a known set
@@ -0,0 +1,74 @@
/**
* Integrations service layer: stored connections (S3 today; MCP/API later) that
* policy sources and pipeline outputs reference by id instead of embedding
* credentials. Secrets are write-only - reads return them masked, and sending
* the mask back on update keeps the stored value.
*/
import { apiClient } from "@portal/api/http";
export type IntegrationType = "S3" | "MCP" | "API";
export type OwnerScope = "USER" | "TEAM" | "SERVER";
/** Mirrors the backend IntegrationConfigResponse; `config` values are masked. */
export interface IntegrationConfig {
id: number;
integrationType: IntegrationType;
name: string;
scope: OwnerScope;
ownerUserId: number | null;
ownerTeamId: number | null;
enabled: boolean;
locked: boolean;
defaultAccess: string;
config: Record<string, unknown>;
canManage: boolean;
createdAt: string;
updatedAt: string;
}
/** Create/update body; omitted fields keep their stored values on update. */
export interface IntegrationConfigRequest {
integrationType?: IntegrationType;
name?: string;
scope?: OwnerScope;
ownerTeamId?: number | null;
enabled?: boolean;
config?: Record<string, unknown>;
}
export async function fetchIntegrations(): Promise<IntegrationConfig[]> {
return apiClient.local.json<IntegrationConfig[]>("/api/v1/integrations");
}
/** The S3 connections the caller may use, for source/output pickers. */
export async function fetchS3Connections(): Promise<IntegrationConfig[]> {
return (await fetchIntegrations()).filter(
(integration) => integration.integrationType === "S3",
);
}
export async function createIntegration(
body: IntegrationConfigRequest,
): Promise<IntegrationConfig> {
return apiClient.local.json<IntegrationConfig>("/api/v1/integrations", {
method: "POST",
body,
});
}
export async function updateIntegration(
id: number,
body: IntegrationConfigRequest,
): Promise<IntegrationConfig> {
return apiClient.local.json<IntegrationConfig>(
`/api/v1/integrations/${encodeURIComponent(id)}`,
{ method: "PUT", body },
);
}
export async function deleteIntegration(id: number): Promise<void> {
await apiClient.local.json<void>(
`/api/v1/integrations/${encodeURIComponent(id)}`,
{ method: "DELETE" },
);
}
+39 -52
View File
@@ -9,9 +9,12 @@
* approach the editor uses for its own catalogue view.
*/
import type { TFunction } from "i18next";
import { apiClient } from "@portal/api/http";
import { fromWirePolicy, toWirePolicy } from "@app/policies/codec";
import { runsToActivity, runsToStats } from "@app/policies/runs";
import { policyStep, type PolicyToolStep } from "@app/policies/operations";
import type { ToolEndpoint } from "@app/types/toolApiTypes";
import type {
PolicyDecodedState,
PolicyRunView,
@@ -65,7 +68,7 @@ export interface PolicyConfigDef {
rules: string[];
scopeLabel: string;
fields: PolicyField[];
defaultOperations: WirePipelineStep[];
defaultOperations: PolicyToolStep[];
}
export interface PolicyState {
@@ -127,20 +130,11 @@ export interface CatalogueEntry {
}
/* ──────────────────────────────────────────────────────────────────────── */
/* Tool → endpoint registry */
/* Endpoint display labels */
/* ──────────────────────────────────────────────────────────────────────── */
export const TOOL_ENDPOINTS: Record<string, string> = {
redact: "/api/v1/security/auto-redact",
sanitize: "/api/v1/security/sanitize-pdf",
watermark: "/api/v1/security/add-watermark",
ocr: "/api/v1/misc/ocr-pdf",
flatten: "/api/v1/misc/flatten",
compress: "/api/v1/misc/compress-pdf",
};
/** Values are i18n keys — render with t(). */
export const ENDPOINT_LABELS: Record<string, string> = {
/** i18n keys keyed by {@link ToolEndpoint}; labels stored steps in the detail view. */
export const ENDPOINT_LABELS: Partial<Record<ToolEndpoint, string>> = {
"/api/v1/security/auto-redact": "portal.policies.endpoints.autoRedact",
"/api/v1/security/sanitize-pdf": "portal.policies.endpoints.sanitizePdf",
"/api/v1/security/add-watermark": "portal.policies.endpoints.addWatermark",
@@ -153,7 +147,8 @@ export function humanizeEndpoint(
path: string,
t: (key: string) => string,
): string {
if (ENDPOINT_LABELS[path]) return t(ENDPOINT_LABELS[path]);
const label = ENDPOINT_LABELS[path as ToolEndpoint];
if (label) return t(label);
const last = path.split("/").filter(Boolean).pop() ?? path;
return last
.replace(/-/g, " ")
@@ -229,10 +224,7 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
"portal.policies.config.ingestion.rules.3",
],
scopeLabel: "portal.policies.config.scopeAll",
defaultOperations: [
{ operation: TOOL_ENDPOINTS.ocr, parameters: {} },
{ operation: TOOL_ENDPOINTS.flatten, parameters: {} },
],
defaultOperations: [policyStep("ocr"), policyStep("flatten")],
fields: [
{
label: "portal.policies.config.ingestion.fields.minConfidence",
@@ -259,32 +251,16 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
],
scopeLabel: "portal.policies.config.scopeAll",
defaultOperations: [
{
operation: TOOL_ENDPOINTS.redact,
parameters: {
mode: "automatic",
useRegex: true,
convertPDFToImage: true,
wordsToRedact: DEFAULT_PII_PATTERNS,
},
},
{
operation: TOOL_ENDPOINTS.sanitize,
parameters: {
removeJavaScript: true,
removeEmbeddedFiles: false,
removeMetadata: false,
removeLinks: false,
removeFonts: false,
},
},
{
operation: TOOL_ENDPOINTS.watermark,
// convertPDFToImage bakes the watermark in so it can't be stripped
parameters: {
convertPDFToImage: true,
},
},
// Flatten to image so redactions can't be lifted off.
policyStep("redact", {
useRegex: true,
convertPDFToImage: true,
wordsToRedact: DEFAULT_PII_PATTERNS,
}),
// JavaScript removal only; the tool enables removeEmbeddedFiles by default, so turn it off.
policyStep("sanitize", { removeEmbeddedFiles: false }),
// Bake in via image so it can't be stripped.
policyStep("watermark", { convertPDFToImage: true }),
],
fields: [],
},
@@ -296,10 +272,7 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
"portal.policies.config.compliance.rules.2",
],
scopeLabel: "portal.policies.config.scopeAll",
defaultOperations: [
{ operation: TOOL_ENDPOINTS.sanitize, parameters: {} },
{ operation: TOOL_ENDPOINTS.flatten, parameters: {} },
],
defaultOperations: [policyStep("sanitize"), policyStep("flatten")],
fields: [
{
label: "portal.policies.config.compliance.fields.frameworks",
@@ -342,7 +315,7 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
"portal.policies.config.routing.rules.2",
],
scopeLabel: "portal.policies.config.scopeAll",
defaultOperations: [{ operation: TOOL_ENDPOINTS.compress, parameters: {} }],
defaultOperations: [policyStep("compress")],
fields: [
{
label: "portal.policies.config.routing.fields.destination",
@@ -373,7 +346,7 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
"portal.policies.config.retention.rules.2",
],
scopeLabel: "portal.policies.config.scopeAll",
defaultOperations: [{ operation: TOOL_ENDPOINTS.compress, parameters: {} }],
defaultOperations: [policyStep("compress")],
fields: [
{
label: "portal.policies.config.retention.fields.keepFor",
@@ -556,17 +529,30 @@ const DEFAULT_RETRY_DELAY = 5;
// POST /api/v1/policies endpoint. The real backend ignores unknown fields.
type CatalogueWireBody = WirePolicy & { categoryId: string };
/**
* The persisted policy name derived from its category, e.g. "Security Policy".
* `category.label` is an i18n key, so translate it before building the name;
* otherwise the raw key is persisted and surfaces in the UI (e.g. the Sources
* "Used by" pill).
*/
function policyDisplayName(entry: CatalogueEntry, t: TFunction): string {
return t("portal.policies.defaultName", {
category: t(entry.category.label),
});
}
/** Build a wire policy from a setup wizard result. */
export function buildWireFromSetup(
entry: CatalogueEntry,
result: PolicySetupResult,
t: TFunction,
enabled = true,
): CatalogueWireBody {
return {
categoryId: entry.category.id,
...toWirePolicy({
id: entry.policy?.state.backendId ?? "",
name: `${entry.category.label} Policy`,
name: policyDisplayName(entry, t),
enabled,
categoryId: entry.category.id,
sources: result.sources,
@@ -589,13 +575,14 @@ export function buildWireFromState(
entry: CatalogueEntry,
policy: DecoratedPolicy,
enabled: boolean,
t: TFunction,
): CatalogueWireBody {
const s = policy.state;
return {
categoryId: entry.category.id,
...toWirePolicy({
id: s.backendId ?? "",
name: `${entry.category.label} Policy`,
name: policyDisplayName(entry, t),
enabled,
categoryId: entry.category.id,
sources: s.sources,
@@ -0,0 +1,176 @@
/**
* Processor-flow assembler for the home visualiser.
*
* Fans in the three real portal surfaces — sources (`/api/v1/sources`), policies
* (`/api/v1/policies`) and their runs (`/api/v1/policies/runs`) — and derives the
* left→middle→right shape the {@link ProcessorFlow} component renders:
*
* sources → policies → outcomes
*
* Everything here is real backend data. Per-run source attribution does not
* exist (a `PolicyRunView` carries `policyId` but no source id), so the flow
* animation is illustrative; the node counts are not — each source's `docs24h`,
* each policy's trailing-24h run count, and the success/failure split all come
* straight from the API.
*/
import { apiClient } from "@portal/api/http";
import { fetchSources } from "@portal/api/sources";
import { POLICY_CATEGORIES } from "@portal/api/policies";
import { fromWirePolicy } from "@app/policies/codec";
import type { PolicyRunView, WirePolicy } from "@app/policies/types";
/** A source that actually feeds the processor today (editor, folder, S3, …). */
export interface FlowSource {
id: string;
/** Display name (already resolved; editor rows get a friendly label). */
name: string;
type: string;
/** Documents this source fed into runs over the trailing 24h. */
docs24h: number;
}
/**
* A connector type shown in the sources column but not yet a real source type —
* a "coming soon" affordance only. `labelKey` is an i18n key.
*/
export interface FlowComingSoonSource {
key: string;
labelKey: string;
}
/**
* Row display state, mirroring the Policies page:
* - `active` — configured + enabled; shows its live 24h run count
* - `off` — available but not set up; offers a "Set up" CTA
* - `locked` — a coming-soon category that doesn't exist yet
*/
export type FlowPolicyState = "active" | "off" | "locked";
/**
* One row in the middle policies column — the full policy catalogue, in the
* same order the Policies page shows, including the coming-soon categories.
*/
export interface FlowPolicy {
/** Category id (also the lane key for the flow animation). */
key: string;
/** i18n key for the category label. */
labelKey: string;
/** Material Symbols icon name (from the catalogue). */
icon: string;
state: FlowPolicyState;
configured: boolean;
runs24h: number;
}
export type FlowOutcomeKey = "success" | "failed";
/** A terminal audit outcome node on the right, counted over the trailing 24h. */
export interface FlowOutcome {
key: FlowOutcomeKey;
labelKey: string;
count24h: number;
}
export interface ProcessorFlow {
sources: FlowSource[];
comingSoonSources: FlowComingSoonSource[];
policies: FlowPolicy[];
outcomes: FlowOutcome[];
}
const DAY_MS = 86_400_000;
/** Connector types the sources column advertises but can't create yet. */
const COMING_SOON_SOURCES: FlowComingSoonSource[] = [
{ key: "apiMcp", labelKey: "portal.processorFlow.sources.comingSoon.apiMcp" },
{
key: "cloud",
labelKey: "portal.processorFlow.sources.comingSoon.cloud",
},
{
key: "email",
labelKey: "portal.processorFlow.sources.comingSoon.email",
},
];
/**
* The full policy catalogue, in the Policies-page order, including the
* coming-soon categories (rendered as locked). `active` rows carry their
* trailing-24h run count.
*/
function buildPolicies(
wirePolicies: WirePolicy[],
runs: PolicyRunView[],
): FlowPolicy[] {
const cutoff = Date.now() - DAY_MS;
const decoded = wirePolicies.map(fromWirePolicy);
return POLICY_CATEGORIES.map((cat) => {
const dp = decoded.find((p) => p.categoryId === cat.id);
const configured = Boolean(dp?.enabled);
const state: FlowPolicyState = configured
? "active"
: cat.comingSoon
? "locked"
: "off";
const runs24h = dp
? runs.filter((r) => r.policyId === dp.id && r.createdAt >= cutoff).length
: 0;
return {
key: cat.id,
labelKey: cat.label,
icon: cat.icon,
state,
configured,
runs24h,
};
});
}
/** Terminal audit outcomes over the trailing 24h — success vs failure. */
function buildOutcomes(runs: PolicyRunView[]): FlowOutcome[] {
const cutoff = Date.now() - DAY_MS;
const recent = runs.filter((r) => r.createdAt >= cutoff);
const success = recent.filter((r) => r.status === "COMPLETED").length;
const failed = recent.filter(
(r) => r.status === "FAILED" || r.status === "CANCELLED",
).length;
return [
{
key: "success",
labelKey: "portal.processorFlow.outcomes.success",
count24h: success,
},
{
key: "failed",
labelKey: "portal.processorFlow.outcomes.failed",
count24h: failed,
},
];
}
/** Assemble the full flow model from the three live portal surfaces. */
export async function fetchProcessorFlow(): Promise<ProcessorFlow> {
const [sourcesResp, wirePolicies, runs] = await Promise.all([
fetchSources(),
apiClient.local.json<WirePolicy[]>("/api/v1/policies"),
apiClient.local
.json<PolicyRunView[]>("/api/v1/policies/runs")
.catch(() => [] as PolicyRunView[]),
]);
const sources: FlowSource[] = sourcesResp.sources.map((s) => ({
id: s.id,
name: s.name,
type: s.type,
docs24h: s.docs24h,
}));
return {
sources,
comingSoonSources: COMING_SOON_SOURCES,
policies: buildPolicies(wirePolicies, runs),
outcomes: buildOutcomes(runs),
};
}
+14 -9
View File
@@ -1,3 +1,7 @@
// The bare i18next singleton (the same instance @app/i18n initializes at
// startup), imported directly so this data module doesn't pull i18n's
// init side effects into unit tests that mock react-i18next.
import i18n from "i18next";
import { apiClient } from "@portal/api/http";
import type { Tier } from "@portal/contexts/TierContext";
@@ -269,22 +273,23 @@ function roleIdFor(u: AdminUserSummaryDto): RoleId {
/** A member's last-seen time as plain language; "Never" when no session is tracked. */
function relativeTime(value: number | string | undefined): string {
if (value === undefined || value === null) return "Never";
if (value === undefined || value === null)
return i18n.t("users.activity.never");
const ts = typeof value === "string" ? Date.parse(value) : value;
if (!Number.isFinite(ts) || ts <= 0) return "Never";
if (!Number.isFinite(ts) || ts <= 0) return i18n.t("users.activity.never");
const mins = Math.max(0, Math.round((Date.now() - ts) / 60000));
if (mins < 1) return "Just now";
if (mins < 60) return `${mins}m ago`;
if (mins < 1) return i18n.t("users.activity.justNow");
if (mins < 60) return i18n.t("users.activity.minutesAgo", { count: mins });
const hours = Math.round(mins / 60);
if (hours < 24) return `${hours}h ago`;
if (hours < 24) return i18n.t("users.activity.hoursAgo", { count: hours });
const days = Math.round(hours / 24);
if (days < 7) return `${days}d ago`;
if (days < 7) return i18n.t("users.activity.daysAgo", { count: days });
const weeks = Math.round(days / 7);
if (weeks < 5) return weeks === 1 ? "1 week ago" : `${weeks} weeks ago`;
if (weeks < 5) return i18n.t("users.activity.weeksAgo", { count: weeks });
const months = Math.round(days / 30);
if (months < 12) return months <= 1 ? "1 month ago" : `${months} months ago`;
if (months < 12) return i18n.t("users.activity.monthsAgo", { count: months });
const years = Math.round(days / 365);
return years <= 1 ? "1 year ago" : `${years} years ago`;
return i18n.t("users.activity.yearsAgo", { count: years });
}
/** 0 / huge sentinel license values mean "no seat limit". */
@@ -15,7 +15,7 @@
align-items: center;
gap: 1.25rem;
padding: 1rem 1.25rem;
background: #16213e;
background: var(--color-hero-navy);
}
.portal-editor-hero__logo {
@@ -123,7 +123,7 @@
.portal-editor-hero__action .portal-editor-hero__cta.sui-btn {
background: #ffffff;
border-color: #ffffff;
color: #16213e;
color: var(--color-hero-navy);
}
.portal-editor-hero__action .portal-editor-hero__cta.sui-btn:hover {
background: rgba(255, 255, 255, 0.88);
@@ -26,7 +26,7 @@ export function ProcessingStatusStrip() {
style={{ background: TIER_INFO[tier].dotColor }}
aria-hidden
/>
{TIER_INFO[tier].label}
{t(TIER_INFO[tier].labelKey)}
</span>
<span className="portal-statusstrip__sep" aria-hidden>
·
@@ -0,0 +1,394 @@
/* Processor-flow visualiser — sources → policies → outcomes. The flow is an
SVG overlay measured from the HTML cards: bézier wires underneath, and a
rAF-driven particle layer on top (see ProcessorFlow.tsx). */
.portal-pf {
--pf-accent: var(--color-green);
}
/* ── Header ──────────────────────────────────────────────────────────────── */
.portal-pf__head {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.75rem;
margin-bottom: 1rem;
}
.portal-pf__head-text {
display: flex;
align-items: baseline;
gap: 0.5rem;
min-width: 0;
}
.portal-pf__head-actions {
display: flex;
align-items: center;
gap: 0.625rem;
flex: none;
}
.portal-pf__live {
align-self: center;
width: 0.5rem;
height: 0.5rem;
border-radius: 50%;
background: var(--color-border);
}
.portal-pf__live--on {
background: var(--pf-accent);
box-shadow: 0 0 0 0 color-mix(in srgb, var(--pf-accent) 55%, transparent);
animation: pf-pulse 2.4s ease-out infinite;
}
.portal-pf__title {
margin: 0;
font-size: 0.9375rem;
font-weight: 600;
color: var(--color-text-1);
}
.portal-pf__connected {
font-size: 0.75rem;
color: var(--color-text-4);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
/* ── Stage: relative box the SVG overlays are measured against ───────────── */
.portal-pf__stage {
position: relative;
padding: 0.25rem 0 0.5rem;
}
.portal-pf__wires,
.portal-pf__particles {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
pointer-events: none;
overflow: visible;
}
.portal-pf__wires {
z-index: 0;
}
.portal-pf__particles {
z-index: 2;
}
.portal-pf__wire-path {
fill: none;
stroke: var(--color-border-light);
stroke-width: 1.25;
}
/* ── Columns ─────────────────────────────────────────────────────────────── */
.portal-pf__cols {
position: relative;
z-index: 1;
display: flex;
justify-content: space-between;
align-items: stretch;
gap: 1rem;
}
.portal-pf__col {
display: flex;
flex-direction: column;
justify-content: center;
gap: 0.5rem;
flex: none;
width: 15rem;
}
.portal-pf__col-head,
.portal-pf__policies-head {
font-size: 0.6875rem;
font-weight: 600;
letter-spacing: 0.04em;
text-transform: uppercase;
color: var(--color-text-5);
margin-bottom: 0.125rem;
}
/* ── Nodes (source + outcome cards) ─────────────────────────────────────────── */
.portal-pf__node {
height: auto;
width: 100%;
text-align: left;
background: var(--color-bg-subtle);
border: 1px solid var(--color-border-light);
border-radius: var(--radius-md);
transition:
background var(--motion-fast),
border-color var(--motion-fast);
}
.portal-pf__node .mantine-Button-label {
flex: 1 1 auto;
justify-content: flex-start;
overflow: visible;
}
.portal-pf__node:hover {
background: var(--color-bg-hover);
border-color: var(--color-border);
}
.portal-pf__node:focus-visible {
outline: 2px solid var(--color-blue);
outline-offset: 2px;
}
.portal-pf__node--soon {
background: transparent;
border-style: dashed;
opacity: 0.75;
}
.portal-pf__node-icon {
display: inline-flex;
align-items: center;
justify-content: center;
flex: none;
width: 1.75rem;
height: 1.75rem;
border-radius: var(--radius-md);
font-size: 0.9375rem;
background: var(--color-bg-muted);
color: var(--color-text-3);
}
.portal-pf__node--soon .portal-pf__node-icon {
color: var(--color-text-5);
font-weight: 600;
}
.portal-pf__node--success .portal-pf__node-icon {
background: var(--color-green-light);
color: var(--color-green-dark);
}
.portal-pf__node--failed .portal-pf__node-icon {
background: var(--color-red-light);
color: var(--color-red-dark);
}
.portal-pf__node-text {
display: flex;
flex-direction: column;
min-width: 0;
}
.portal-pf__node-text strong {
font-size: 0.8125rem;
font-weight: 600;
color: var(--color-text-1);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.portal-pf__node-text span {
font-size: 0.6875rem;
color: var(--color-text-4);
}
/* ── Policies card (core) ───────────────────────────────────────────────────── */
.portal-pf__policies {
align-self: center;
flex: none;
width: 18rem;
padding: 0.75rem;
background: var(--color-bg-subtle);
border: 1px solid var(--color-border-light);
border-radius: var(--radius-lg);
box-shadow: var(--shadow-sm);
}
.portal-pf__policies-head {
display: flex;
align-items: center;
justify-content: space-between;
padding: 0 0.25rem 0.5rem;
}
.portal-pf__policies-active {
text-transform: none;
letter-spacing: 0;
color: var(--color-green-dark);
}
.portal-pf__policy {
padding: 0.5rem 0.25rem;
}
.portal-pf__policy + .portal-pf__policy {
border-top: 1px solid var(--color-border-light);
}
.portal-pf__policy-line {
display: flex;
align-items: center;
gap: 0.5rem;
}
.portal-pf__policy-icon {
display: inline-flex;
align-items: center;
justify-content: center;
flex: none;
width: 1.5rem;
height: 1.5rem;
border-radius: 6px;
font-size: 0.875rem;
color: var(--color-text-4);
transition:
color 0.3s ease,
background-color 0.3s ease,
box-shadow 0.3s ease;
}
.portal-pf__policy--active .portal-pf__policy-icon {
color: var(--pf-accent);
}
.portal-pf__policy--locked .portal-pf__policy-icon {
color: var(--color-text-5);
}
/* Leading-LED blink: added for 150ms as a particle threads this row's lane,
then eased back out by the transition above. */
.portal-pf__policy-icon.is-pulse {
color: var(--color-green-dark);
background-color: var(--color-green-light);
box-shadow: 0 0 8px 1px
color-mix(in srgb, var(--color-green) 55%, transparent);
}
.portal-pf__policy-label {
flex: 1 1 auto;
font-size: 0.8125rem;
font-weight: 600;
color: var(--color-text-2);
min-width: 0;
}
.portal-pf__policy--active .portal-pf__policy-label {
color: var(--color-text-1);
}
.portal-pf__policy--locked .portal-pf__policy-label {
color: var(--color-text-4);
font-weight: 500;
}
.portal-pf__policy-count {
font-size: 0.75rem;
color: var(--color-text-4);
font-variant-numeric: tabular-nums;
white-space: nowrap;
}
.portal-pf__policy-soon {
flex: none;
font-size: 0.6875rem;
font-weight: 600;
color: var(--color-text-5);
text-transform: uppercase;
letter-spacing: 0.04em;
}
/* Vertical padding via the `py` prop; height auto so it grows with it. */
.portal-pf__setup {
flex: none;
height: auto;
}
/* ── Footnote / loading ─────────────────────────────────────────────────────── */
.portal-pf__foot {
margin: 0.875rem 0 0;
font-size: 0.6875rem;
color: var(--color-text-5);
}
.portal-pf__loading {
padding: 0.5rem 0;
}
/* ── Sankey lens ────────────────────────────────────────────────────────────── */
.portal-pf__sankey {
max-width: 46rem;
margin: 0.5rem auto 0;
}
.portal-pf__sankey svg {
display: block;
width: 100%;
height: auto;
overflow: visible;
}
.portal-pf__sankey-label {
font-size: 12px;
font-weight: 600;
fill: var(--color-text-2);
font-variant-numeric: tabular-nums;
}
.portal-pf__sankey-caption {
font-size: 10.5px;
font-weight: 600;
letter-spacing: 0.05em;
text-transform: uppercase;
fill: var(--color-text-5);
}
.portal-pf__sankey-empty {
display: flex;
align-items: center;
justify-content: center;
min-height: 11rem;
}
@keyframes pf-pulse {
0% {
box-shadow: 0 0 0 0 color-mix(in srgb, var(--pf-accent) 55%, transparent);
}
70% {
box-shadow: 0 0 0 0.4rem transparent;
}
100% {
box-shadow: 0 0 0 0 transparent;
}
}
/* ── Responsive: stack the columns; the measured-geometry flow overlay only
makes sense on the wide 3-column layout, so drop it below the breakpoint. ── */
@media (max-width: 60rem) {
.portal-pf__wires,
.portal-pf__particles {
display: none;
}
.portal-pf__cols {
flex-direction: column;
gap: 0.75rem;
}
.portal-pf__col,
.portal-pf__policies {
width: 100%;
align-self: stretch;
}
}
@media (prefers-reduced-motion: reduce) {
.portal-pf__live--on {
animation: none;
}
}
@@ -0,0 +1,60 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { http, HttpResponse } from "msw";
import { ProcessorFlow } from "@portal/components/ProcessorFlow";
/**
* The home processor visualiser. Data is served by the global portal MSW
* handlers (seeded sources + one active Security policy + its runs), so the
* middle column shows Security "active" and Classification with a "Set up" CTA,
* and the outcomes reflect the seeded 24h success/failure split.
*
* NB: the flow particles animate via requestAnimationFrame, which browsers
* pause while the tab/preview is hidden — open the story in a focused tab to
* see the dots move.
*/
const meta: Meta<typeof ProcessorFlow> = {
title: "Portal/Components/ProcessorFlow",
component: ProcessorFlow,
parameters: { layout: "padded" },
};
export default meta;
type Story = StoryObj<typeof ProcessorFlow>;
/** Live machine: Security configured + real throughput → the flow runs. */
export const Default: Story = {};
/**
* Nothing set up and no activity — the empty state from the design. In
* production the flow stays still here; the DEV_KEEP_FLOWING dev flag forces it
* on with synthetic rates so the animation is visible while iterating.
*/
export const IdleEmpty: Story = {
parameters: {
msw: {
handlers: [
http.get("/api/v1/sources", () =>
HttpResponse.json({
kpis: [],
sources: [
{
id: "editor",
name: "Editor",
type: "editor",
status: "active",
referenceCount: 0,
referencingPolicies: [],
config: [],
docsTotal: 0,
docs24h: 0,
docs30d: 0,
},
],
}),
),
http.get("/api/v1/policies", () => HttpResponse.json([])),
http.get("/api/v1/policies/runs", () => HttpResponse.json([])),
],
},
},
};
@@ -0,0 +1,184 @@
import { useState } from "react";
import { useNavigate } from "react-router-dom";
import { useTranslation } from "react-i18next";
import { Card, SegmentedControl, Skeleton, StatusBadge } from "@app/ui";
import {
useView,
VIEW_PATHS,
toPortalPath,
} from "@portal/contexts/ViewContext";
import { useAsync } from "@portal/hooks/useAsync";
import {
fetchProcessorFlow,
type ProcessorFlow as ProcessorFlowModel,
} from "@portal/api/processorFlow";
import {
DEV_KEEP_FLOWING,
DEV_SYNTH_RATE,
type Lens,
} from "@portal/components/processor-flow/flowTypes";
import { useFlowGeometry } from "@portal/components/processor-flow/useFlowGeometry";
import { useFlowParticles } from "@portal/components/processor-flow/useFlowParticles";
import { FlowSources } from "@portal/components/processor-flow/FlowSources";
import { FlowPolicies } from "@portal/components/processor-flow/FlowPolicies";
import { FlowOutcomes } from "@portal/components/processor-flow/FlowOutcomes";
import { FlowSankey } from "@portal/components/processor-flow/FlowSankey";
import "@portal/components/ProcessorFlow.css";
/**
* Animated processor visualiser for the home surface: connected sources on the
* left flow through the standing policies in the middle to their audit outcomes
* on the right. Two lenses — a live particle flow and a Sankey summary.
*
* This module wires the data + gating together; the moving parts live under
* `processor-flow/`: geometry ({@link useFlowGeometry}), the rAF particle loop
* ({@link useFlowParticles}), the three columns, and the Sankey. The flow only
* runs when something is set up AND there's activity; an idle machine stays
* still (unless {@link DEV_KEEP_FLOWING} forces it while iterating).
*/
export function ProcessorFlow() {
const { t } = useTranslation();
const { setActiveView } = useView();
const navigate = useNavigate();
const { data, loading } = useAsync<ProcessorFlowModel>(
() => fetchProcessorFlow(),
[],
);
const [lens, setLens] = useState<Lens>("flow");
const isLoading = loading && data === null;
/** Deep-link to the Policies page and auto-open that policy's setup wizard. */
const openPolicySetup = (key: string) =>
navigate(
`${toPortalPath(VIEW_PATHS.policies)}?setup=${encodeURIComponent(key)}`,
);
/** Deep-link to Infrastructure with the audit-log tab open. */
const openAuditLog = () =>
navigate(`${toPortalPath(VIEW_PATHS.infrastructure)}?tab=audit`);
const sources = data?.sources ?? [];
const policies = data?.policies ?? [];
const outcomes = data?.outcomes ?? [];
const comingSoonSources = data?.comingSoonSources ?? [];
// ── Flow gating: run only when something is set up AND there's activity.
const totalRate = sources.reduce((sum, s) => sum + s.docs24h, 0);
const hasConfigured = policies.some((p) => p.configured);
const liveFlow = hasConfigured && totalRate > 0;
// When forcing for dev with no live flow, synthesise rates + thread every row.
const devForced = DEV_KEEP_FLOWING && !liveFlow;
const animate = liveFlow || devForced;
// Particles only thread configured (active) policies; while dev-forcing with
// no live flow, thread the available (non-locked) rows so the demo has lanes.
const laneKeys = policies
.filter((p) => (devForced ? p.state !== "locked" : p.state === "active"))
.map((p) => p.key);
const activeCount = policies.filter((p) => p.state === "active").length;
const pdfsProcessed = outcomes.reduce((sum, o) => sum + o.count24h, 0);
const statsLabel = t("portal.processorFlow.stats", {
connected: sources.length,
processed: pdfsProcessed.toLocaleString(),
});
// Per-source rates + outcome weights feeding the particle loop.
const rates = sources.map((s) => (devForced ? DEV_SYNTH_RATE : s.docs24h));
const weights = (() => {
const raw = outcomes.map((o) => o.count24h);
const sum = raw.reduce((a, b) => a + b, 0);
if (sum > 0) return raw.map((v) => v / sum);
// No real outcomes yet (dev flow): success-heavy default.
return outcomes.map((o) => (o.key === "failed" ? 0.15 : 0.85));
})();
const outcomeKeys = outcomes.map((o) => o.key);
const { wrapRef, srcRefs, outRefs, coreRef, laneRefs, geoRef, wires } =
useFlowGeometry();
const pGroupRef = useFlowParticles({
geoRef,
animate,
lens,
rates,
weights,
laneKeys,
outcomeKeys,
});
return (
<Card padding="loose" className="portal-pf">
<header className="portal-pf__head">
<div className="portal-pf__head-text">
<span
className={
"portal-pf__live" + (animate ? " portal-pf__live--on" : "")
}
aria-hidden
/>
<h2 className="portal-pf__title">
{t("portal.processorFlow.title")}
</h2>
<span className="portal-pf__connected">{statsLabel}</span>
</div>
<div className="portal-pf__head-actions">
<StatusBadge tone={animate ? "success" : "neutral"} size="sm">
{t("portal.processorFlow.liveBadge")}
</StatusBadge>
<SegmentedControl<Lens>
size="xs"
value={lens}
onChange={setLens}
ariaLabel={t("portal.processorFlow.lens.ariaLabel")}
options={[
{ label: t("portal.processorFlow.lens.flow"), value: "flow" },
{ label: t("portal.processorFlow.lens.sankey"), value: "sankey" },
]}
/>
</div>
</header>
{isLoading ? (
<div className="portal-pf__loading" aria-hidden>
<Skeleton height="9rem" />
</div>
) : lens === "sankey" ? (
<FlowSankey sources={sources} outcomes={outcomes} policies={policies} />
) : (
<div className="portal-pf__stage" ref={wrapRef}>
<svg className="portal-pf__wires" aria-hidden>
{wires}
</svg>
<div className="portal-pf__cols">
<FlowSources
sources={sources}
comingSoonSources={comingSoonSources}
srcRefs={srcRefs}
onOpen={() => setActiveView("sources")}
/>
<FlowPolicies
policies={policies}
activeCount={activeCount}
coreRef={coreRef}
laneRefs={laneRefs}
onSetup={openPolicySetup}
/>
<FlowOutcomes
outcomes={outcomes}
outRefs={outRefs}
onOpen={openAuditLog}
/>
</div>
<svg className="portal-pf__particles" aria-hidden>
<g ref={pGroupRef} />
</svg>
</div>
)}
<p className="portal-pf__foot">{t("portal.processorFlow.footnote")}</p>
</Card>
);
}
@@ -18,7 +18,7 @@
gap: 1rem;
flex-wrap: wrap;
padding: 0.875rem 1.25rem;
background: #16213e;
background: var(--color-hero-navy);
}
.portal-welcome__brand {
@@ -90,7 +90,7 @@
.portal-welcome__header .portal-welcome__cta.sui-btn {
background: #ffffff;
border-color: #ffffff;
color: #16213e;
color: var(--color-hero-navy);
}
.portal-welcome__header .portal-welcome__cta.sui-btn:hover {
background: rgba(255, 255, 255, 0.88);
@@ -1,7 +1,11 @@
import { useState } from "react";
import { useTranslation } from "react-i18next";
import { StatusBadge, Tabs, type TabItem } from "@app/ui";
import { type Agent, AGENT_STATUS_TONE } from "@portal/api/agents";
import {
type Agent,
AGENT_STATUS_LABEL,
AGENT_STATUS_TONE,
} from "@portal/api/agents";
import { ScenariosPanel } from "@portal/components/agent-builder/ScenariosPanel";
import { ToolsPanel } from "@portal/components/agent-builder/ToolsPanel";
import { EvalsPanel } from "@portal/components/agent-builder/EvalsPanel";
@@ -52,7 +56,7 @@ export function AgentBuilderPanel({
</div>
<div className="portal-agents__builder-meta">
<StatusBadge tone={AGENT_STATUS_TONE[agent.status]} size="sm">
{agent.status}
{t(AGENT_STATUS_LABEL[agent.status])}
</StatusBadge>
<code className="portal-agents__builder-version">
{agent.version}
@@ -1,5 +1,9 @@
import { useTranslation } from "react-i18next";
import { type Agent, AGENT_STATUS_TONE } from "@portal/api/agents";
import {
type Agent,
AGENT_STATUS_LABEL,
AGENT_STATUS_TONE,
} from "@portal/api/agents";
import { Button, StatusBadge } from "@app/ui";
import "@portal/views/AgentBuilder.css";
@@ -40,7 +44,7 @@ export function AgentSelector({
</span>
<span className="portal-agents__selector-meta">
<StatusBadge tone={AGENT_STATUS_TONE[a.status]} size="sm">
{a.status}
{t(AGENT_STATUS_LABEL[a.status])}
</StatusBadge>
<code className="portal-agents__selector-version">{a.version}</code>
</span>
@@ -1,6 +1,10 @@
import { useTranslation } from "react-i18next";
import { Button, StatusBadge } from "@app/ui";
import { type Agent, AGENT_STATUS_TONE } from "@portal/api/agents";
import {
type Agent,
AGENT_STATUS_LABEL,
AGENT_STATUS_TONE,
} from "@portal/api/agents";
import "@portal/views/AgentBuilder.css";
interface VersionsPanelProps {
@@ -52,7 +56,7 @@ export function VersionsPanel({ agent, historyUnlocked }: VersionsPanelProps) {
{v.version}
</code>
<StatusBadge tone={AGENT_STATUS_TONE[v.status]} size="sm">
{v.status}
{t(AGENT_STATUS_LABEL[v.status])}
</StatusBadge>
{isCurrent && (
<StatusBadge tone="info" size="sm" showDot={false}>
@@ -12,9 +12,11 @@ import {
} from "@app/ui";
import {
type SdkComponent,
BILLING_UNIT_LABEL,
MATURITY_META,
formatPrice,
} from "@portal/api/sdkComponents";
import { TIER_INFO } from "@portal/contexts/TierContext";
import { ComponentPropsTable } from "@portal/components/catalogue/ComponentPropsTable";
import "@portal/views/Components.css";
@@ -113,7 +115,7 @@ export function ComponentDetailModal({
title={t("portal.catalogue.detail.locked.title")}
description={t("portal.catalogue.detail.locked.description", {
name: component.name,
tier: component.minTier,
plan: t(TIER_INFO[component.minTier].labelKey),
})}
/>
)}
@@ -205,7 +207,7 @@ export function ComponentDetailModal({
/>
<StatTile
label={t("portal.catalogue.detail.stats.billedOn")}
value={component.pricing.unit}
value={t(BILLING_UNIT_LABEL[component.pricing.unit])}
/>
<StatTile
label={t("portal.catalogue.detail.stats.freeQuota")}
@@ -220,7 +222,7 @@ export function ComponentDetailModal({
</div>
<p className="portal-components__pricing-note">
{t("portal.catalogue.detail.pricing.note", {
unit: component.pricing.unit,
unit: t(BILLING_UNIT_LABEL[component.pricing.unit]),
})}
</p>
</div>
@@ -157,7 +157,7 @@ export function ReviewQueue({ documents, loading }: ReviewQueueProps) {
<Button
variant="secondary"
onClick={() =>
navigate(`${toPortalPath(VIEW_PATHS.sources)}?new`)
navigate(`${toPortalPath(VIEW_PATHS.sources)}/new`)
}
>
{t("portal.documents.queue.empty.connectSource")}
@@ -23,7 +23,7 @@ export function ApiKeyCard({ apiKey }: { apiKey: ApiKey }) {
rightSection={
<span className="portal-infra__key-head-right">
<StatusBadge tone={KEY_TONE[apiKey.status]} size="sm">
{KEY_LABEL[apiKey.status]}
{t(KEY_LABEL[apiKey.status])}
</StatusBadge>
<span
className={"portal-infra__chevron" + (open ? " is-open" : "")}
@@ -0,0 +1,52 @@
import { describe, expect, it, vi } from "vitest";
import { render, screen } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { Tooltip } from "@app/components/shared/Tooltip";
import type { ToolRegistry } from "@app/data/toolsTaxonomy";
import type { WorkingToolStep } from "@app/hooks/tools/shared/toolAutomation";
import { PipelineStepSettings } from "@portal/components/pipelines/PipelineStepSettings";
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string, fallback?: string) => fallback ?? key,
}),
}));
// A stand-in tool-settings UI that uses the shared editor Tooltip. The Tooltip
// pulls in the Preferences + Sidebar contexts, which the portal does not mount
// app-wide — so this reproduces the "usePreferences must be used within a
// PreferencesProvider" crash unless PipelineStepSettings supplies them.
function TooltipSettings() {
return (
<Tooltip content="help">
<button type="button">field</button>
</Tooltip>
);
}
const step = {
support: "editable",
toolId: "compress",
params: {},
} as unknown as WorkingToolStep;
const registry = {
compress: { automationSettings: TooltipSettings },
} as unknown as Partial<ToolRegistry>;
describe("PipelineStepSettings", () => {
it("renders reused editor tool settings (which use the shared Tooltip) without app-wide Preferences/Sidebar providers", () => {
expect(() =>
render(
<MantineProvider>
<PipelineStepSettings
step={step}
registry={registry}
onChange={() => {}}
/>
</MantineProvider>,
),
).not.toThrow();
expect(screen.getByText("field")).toBeInTheDocument();
});
});
@@ -1,6 +1,8 @@
import { Suspense } from "react";
import { useTranslation } from "react-i18next";
import { Banner } from "@app/ui";
import { PreferencesProvider } from "@app/contexts/PreferencesContext";
import { SidebarProvider } from "@app/contexts/SidebarContext";
import { type ToolRegistry } from "@app/data/toolsTaxonomy";
import { type ErasedToolParams } from "@app/hooks/tools/shared/toolOperationTypes";
import { type WorkingToolStep } from "@app/hooks/tools/shared/toolAutomation";
@@ -46,14 +48,18 @@ export function PipelineStepSettings({
}
return (
<Suspense fallback={null}>
<Settings
parameters={step.params}
onParameterChange={(key, value) =>
onChange({ ...step.params, [key]: value })
}
disabled={false}
/>
</Suspense>
<PreferencesProvider>
<SidebarProvider>
<Suspense fallback={null}>
<Settings
parameters={step.params}
onParameterChange={(key, value) =>
onChange({ ...step.params, [key]: value })
}
disabled={false}
/>
</Suspense>
</SidebarProvider>
</PreferencesProvider>
);
}
@@ -0,0 +1,134 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
fireEvent,
render as baseRender,
screen,
waitFor,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { PolicySetupWizard } from "@portal/components/policies/PolicySetupWizard";
import {
POLICY_CATEGORIES,
POLICY_CONFIG,
type CatalogueEntry,
type DecoratedPolicy,
type PolicySetupResult,
type PipelineStep,
} from "@portal/api/policies";
const render = (ui: Parameters<typeof baseRender>[0]) =>
baseRender(ui, { wrapper: MantineProvider });
// Deterministic i18n: return the fallback when given, else the key. initReactI18next is stubbed
// because the import graph pulls core/i18n.ts, which registers it as a plugin.
vi.mock("react-i18next", () => ({
useTranslation: () => ({
// Second arg is a string fallback in some call sites and an interpolation object in others;
// only treat a string as the fallback.
t: (key: string, fallback?: unknown) =>
typeof fallback === "string" ? fallback : key,
i18n: { changeLanguage: vi.fn() },
}),
initReactI18next: { type: "3rdParty", init: vi.fn() },
}));
const fetchSources = vi.fn();
vi.mock("@portal/api/sources", () => ({
fetchSources: () => fetchSources(),
}));
const CONTINUE = "portal.policies.wizard.actions.continue";
const SAVE_CHANGES = "portal.policies.wizard.actions.saveChanges";
const ENABLE = "portal.policies.wizard.actions.enablePolicy";
const security = POLICY_CATEGORIES.find((c) => c.id === "security")!;
const securityConfig = POLICY_CONFIG.security;
function editEntry(steps: PipelineStep[]): CatalogueEntry {
const policy: DecoratedPolicy = {
category: security,
config: securityConfig,
state: {
configured: true,
status: "active",
sources: ["editor"],
scopeTypes: [],
reviewerEmail: "",
fieldValues: {},
runOn: "upload",
outputMode: "new_version",
outputName: "",
outputNamePosition: "suffix",
maxRetries: 0,
retryDelayMinutes: 0,
backendId: "pol-1",
isDefault: true,
},
steps,
stats: { enforced: 0, dataProcessed: "-", activeFor: "-" },
activity: [],
};
return { category: security, config: securityConfig, policy };
}
/** Advance the wizard from the workflow tab to the settings tab and submit. */
async function submitWizard(saveLabel: string) {
fireEvent.click(await screen.findByRole("button", { name: CONTINUE }));
fireEvent.click(await screen.findByRole("button", { name: saveLabel }));
}
describe("PolicySetupWizard", () => {
beforeEach(() => {
fetchSources.mockResolvedValue({ sources: [] });
});
it("round-trips a saved step's backend params on edit", async () => {
const onSubmit = vi.fn().mockResolvedValue(undefined);
const entry = editEntry([
{
operation: "/api/v1/security/auto-redact",
parameters: { listOfText: "foo\nbar", useRegex: true },
},
]);
render(
<PolicySetupWizard entry={entry} onClose={vi.fn()} onSubmit={onSubmit} />,
);
await submitWizard(SAVE_CHANGES);
await waitFor(() => expect(onSubmit).toHaveBeenCalled());
const result = onSubmit.mock.calls[0][1] as PolicySetupResult;
// Only the saved tool is enabled on edit, and its patterns survive the wire -> UI -> wire trip.
expect(result.steps).toEqual([
expect.objectContaining({
operation: "/api/v1/security/auto-redact",
parameters: expect.objectContaining({ listOfText: "foo\nbar" }),
}),
]);
});
it("seeds the preset chain for a new policy (redact + sanitize on, watermark off)", async () => {
const onSubmit = vi.fn().mockResolvedValue(undefined);
const entry: CatalogueEntry = {
category: security,
config: securityConfig,
policy: null,
};
render(
<PolicySetupWizard entry={entry} onClose={vi.fn()} onSubmit={onSubmit} />,
);
await submitWizard(ENABLE);
await waitFor(() => expect(onSubmit).toHaveBeenCalled());
const result = onSubmit.mock.calls[0][1] as PolicySetupResult;
const endpoints = result.steps.map((s) => s.operation);
expect(endpoints).toEqual([
"/api/v1/security/auto-redact",
"/api/v1/security/sanitize-pdf",
]);
// Redact carries the preset PII patterns as the backend's listOfText.
const redact = result.steps[0].parameters as { listOfText?: string };
expect(redact.listOfText).toBeTruthy();
});
});
@@ -13,23 +13,24 @@ import {
} from "@app/ui";
import { SettingsRow } from "@app/ui/SettingsRow";
import {
TOOL_ENDPOINTS,
humanizeEndpoint,
type CatalogueEntry,
type PipelineStep,
type PolicySetupResult,
} from "@portal/api/policies";
import type { ToolRegistry, ToolRegistryEntry } from "@app/data/toolsTaxonomy";
import {
deserializeToolStep,
serializeStepFromEndpoint,
} from "@app/hooks/tools/shared/toolAutomation";
policyEndpoint,
policyStepFromWire,
policyStepToWire,
type PolicyParams,
type PolicyToolId,
type PolicyToolStep,
} from "@app/policies/operations";
import { fetchSources } from "@portal/api/sources";
import { useAsync } from "@portal/hooks/useAsync";
import { PolicyFieldRow } from "@portal/components/policies/PolicyFieldRow";
import { policyIcon } from "@portal/components/policies/policyIcons";
import { sourceTypeMeta } from "@portal/components/sources/sourceTypes";
import { useToolRegistry } from "@app/contexts/ToolRegistryContext";
import { PolicyRedactConfig } from "@app/components/policies/PolicyRedactConfig";
import { PolicyWatermarkConfig } from "@app/components/policies/PolicyWatermarkConfig";
import "@portal/views/Policies.css";
@@ -47,12 +48,8 @@ interface PolicySetupWizardProps {
type Step = "workflow" | "settings";
/** A configurable tool in the workflow step: whether it runs + its params. */
interface ToolState {
operation: string;
enabled: boolean;
parameters: Record<string, unknown>;
}
/** A policy step plus whether it runs. */
type ToolState = PolicyToolStep & { enabled: boolean };
/** Resolve each field's effective value: saved override, else definition default. */
function resolveFieldValues(
@@ -69,9 +66,8 @@ function resolveFieldValues(
* round-trips); otherwise the category preset's default chain. Each preset step
* starts enabled — the user toggles tools off in the workflow.
*/
// Temporary: tracks which tools start disabled until the tool registry lands in
// the portal and can drive this via registry metadata or a defaultEnabled flag.
const DISABLED_BY_DEFAULT = new Set(["/api/v1/security/add-watermark"]);
// Temporary until the catalogue carries a defaultEnabled flag.
const DISABLED_BY_DEFAULT = new Set<PolicyToolId>(["watermark"]);
/**
* Policy-facing framing for each capability a policy can include. Labels and
@@ -81,43 +77,43 @@ const DISABLED_BY_DEFAULT = new Set(["/api/v1/security/add-watermark"]);
* the humanised endpoint name with no description.
*/
const CAPABILITY_META: Record<
string,
PolicyToolId,
{ labelKey: string; labelEn: string; descKey: string; descEn: string }
> = {
[TOOL_ENDPOINTS.redact]: {
redact: {
labelKey: "portal.policies.wizard.capability.redact.label",
labelEn: "Redact sensitive information",
descKey: "portal.policies.wizard.capability.redact.desc",
descEn:
"Finds and blacks out sensitive details — like Social Security and card numbers — so they can't be read.",
},
[TOOL_ENDPOINTS.sanitize]: {
sanitize: {
labelKey: "portal.policies.wizard.capability.sanitize.label",
labelEn: "Strip active content",
descKey: "portal.policies.wizard.capability.sanitize.desc",
descEn:
"Removes hidden JavaScript so nothing can run automatically when the document is opened.",
},
[TOOL_ENDPOINTS.watermark]: {
watermark: {
labelKey: "portal.policies.wizard.capability.watermark.label",
labelEn: "Apply a watermark",
descKey: "portal.policies.wizard.capability.watermark.desc",
descEn: "Stamps a visible mark (e.g. “Confidential”) across every page.",
},
[TOOL_ENDPOINTS.ocr]: {
ocr: {
labelKey: "portal.policies.wizard.capability.ocr.label",
labelEn: "Make text searchable",
descKey: "portal.policies.wizard.capability.ocr.desc",
descEn: "Runs OCR so scanned pages become selectable, searchable text.",
},
[TOOL_ENDPOINTS.flatten]: {
flatten: {
labelKey: "portal.policies.wizard.capability.flatten.label",
labelEn: "Flatten the document",
descKey: "portal.policies.wizard.capability.flatten.desc",
descEn:
"Merges form fields and annotations into the page so they can't be edited.",
},
[TOOL_ENDPOINTS.compress]: {
compress: {
labelKey: "portal.policies.wizard.capability.compress.label",
labelEn: "Reduce file size",
descKey: "portal.policies.wizard.capability.compress.desc",
@@ -125,29 +121,24 @@ const CAPABILITY_META: Record<
},
};
function seedTools(
entry: CatalogueEntry,
registry: Partial<ToolRegistry>,
): ToolState[] {
function seedTools(entry: CatalogueEntry): ToolState[] {
const savedSteps = entry.policy?.steps ?? [];
const savedByOp = new Map(savedSteps.map((s) => [s.operation, s]));
// Always use defaultOperations as the canonical list so tools added after a
// policy was first saved still appear when editing.
return entry.config.defaultOperations.map((s) => {
const saved = savedByOp.get(s.operation);
const savedByTool = new Map<PolicyToolId, PolicyToolStep>();
for (const wire of savedSteps) {
const step = policyStepFromWire(wire);
if (step) savedByTool.set(step.toolId, step);
}
// defaultOperations is the canonical list (so tools added later still show on edit); a saved
// step's params win over the preset.
return entry.config.defaultOperations.map((preset) => {
const saved = savedByTool.get(preset.toolId);
return {
operation: s.operation,
...(saved ?? preset),
enabled: saved
? true
: savedSteps.length > 0
? false
: !DISABLED_BY_DEFAULT.has(s.operation),
// Saved steps are in the backend contract shape; map them back to the UI
// shape the config controls edit (e.g. `listOfText` -> `wordsToRedact`).
// Presets are already authored in the UI shape, so use them as-is.
parameters: saved
? deserializeToolStep(saved, registry).params
: s.parameters,
: !DISABLED_BY_DEFAULT.has(preset.toolId),
};
});
}
@@ -185,26 +176,12 @@ function PolicySetupWizardBody({
onSubmit: (entry: CatalogueEntry, result: PolicySetupResult) => Promise<void>;
}) {
const { t } = useTranslation();
const { allTools: toolRegistry } = useToolRegistry();
// Portal tool operations are endpoint paths (/api/v1/…), not short registry IDs.
// Build a reverse map so we can look up icons and display names by endpoint.
const registryByEndpoint = useMemo(() => {
const map = new Map<string, ToolRegistryEntry>();
for (const entry of Object.values(toolRegistry)) {
const ep = (entry as ToolRegistryEntry).operationConfig?.endpoint;
if (typeof ep === "string") map.set(ep, entry as ToolRegistryEntry);
}
return map;
}, [toolRegistry]);
const { category, config, policy } = entry;
const isEdit = policy != null;
const [step, setStep] = useState<Step>("workflow");
const [tools, setTools] = useState<ToolState[]>(() =>
seedTools(entry, toolRegistry),
);
const [tools, setTools] = useState<ToolState[]>(() => seedTools(entry));
const [fieldValues, setFieldValues] = useState(() =>
resolveFieldValues(entry),
);
@@ -213,22 +190,11 @@ function PolicySetupWizardBody({
);
const sourcesAsync = useAsync(() => fetchSources(), []);
const availableSources = useMemo(() => {
const backendSources = (sourcesAsync.data?.sources ?? []).filter(
(s) => s.status !== "disabled",
);
const editorSource = {
id: "editor",
name: t("portal.sources.types.editor.label"),
type: "editor",
status: "active" as const,
referenceCount: 0,
referencingPolicies: [],
config: [],
docsTotal: null,
};
return [editorSource, ...backendSources];
}, [sourcesAsync.data, t]);
const availableSources = useMemo(
() =>
(sourcesAsync.data?.sources ?? []).filter((s) => s.status !== "disabled"),
[sourcesAsync.data],
);
// Document-type scoping has no UI; preserve any saved scope on edit and
// default new policies to all document types.
const [scopeTypes] = useState<string[]>(policy?.state.scopeTypes ?? []);
@@ -256,9 +222,20 @@ function PolicySetupWizardBody({
const enabledTools = useMemo(() => tools.filter((tl) => tl.enabled), [tools]);
function patchTool(operation: string, patch: Partial<ToolState>) {
function setToolEnabled(toolId: PolicyToolId, enabled: boolean) {
setTools((prev) =>
prev.map((tl) => (tl.operation === operation ? { ...tl, ...patch } : tl)),
prev.map((tl) => (tl.toolId === toolId ? { ...tl, enabled } : tl)),
);
}
function setToolParams<Id extends PolicyToolId>(
toolId: Id,
params: PolicyParams<Id>,
) {
setTools((prev) =>
prev.map((tl) =>
tl.toolId === toolId ? ({ ...tl, params } as ToolState) : tl,
),
);
}
@@ -277,11 +254,8 @@ function PolicySetupWizardBody({
}
setError(null);
setSubmitting(true);
// Map each tool's UI-shaped params (e.g. redact's `wordsToRedact`) into the
// backend step contract (e.g. `listOfText`) via its `toApiParams`; saving the
// UI shape verbatim would drop those fields and the step would run with none.
const steps: PipelineStep[] = enabledTools.map((tl) =>
serializeStepFromEndpoint(tl.operation, tl.parameters, toolRegistry),
policyStepToWire(tl),
);
try {
await onSubmit(entry, {
@@ -386,20 +360,16 @@ function PolicySetupWizardBody({
<Card padding="none">
<div className="portal-policies__capabilities">
{tools.map((tl) => {
const meta = CAPABILITY_META[tl.operation];
const meta = CAPABILITY_META[tl.toolId];
const label = meta
? t(meta.labelKey, meta.labelEn)
: (registryByEndpoint.get(tl.operation)?.name ??
humanizeEndpoint(tl.operation, t));
: humanizeEndpoint(policyEndpoint(tl.toolId), t);
const description = meta
? t(meta.descKey, meta.descEn)
: undefined;
const hasConfig =
tl.operation === TOOL_ENDPOINTS.redact ||
tl.operation === TOOL_ENDPOINTS.watermark;
return (
<div
key={tl.operation}
key={tl.toolId}
className="portal-policies__capability"
data-on={tl.enabled || undefined}
>
@@ -411,27 +381,27 @@ function PolicySetupWizardBody({
size="sm"
checked={tl.enabled}
onChange={(checked) =>
patchTool(tl.operation, { enabled: checked })
setToolEnabled(tl.toolId, checked)
}
label=""
/>
}
/>
{tl.enabled && hasConfig && (
{tl.enabled && (
<div className="portal-policies__capability-config">
{tl.operation === TOOL_ENDPOINTS.redact && (
{tl.toolId === "redact" && (
<PolicyRedactConfig
parameters={tl.parameters}
onChange={(parameters) =>
patchTool(tl.operation, { parameters })
parameters={tl.params}
onChange={(params) =>
setToolParams("redact", params)
}
/>
)}
{tl.operation === TOOL_ENDPOINTS.watermark && (
{tl.toolId === "watermark" && (
<PolicyWatermarkConfig
parameters={tl.parameters}
onChange={(parameters) =>
patchTool(tl.operation, { parameters })
parameters={tl.params}
onChange={(params) =>
setToolParams("watermark", params)
}
/>
)}
@@ -475,9 +445,8 @@ function PolicySetupWizardBody({
{t("portal.policies.wizard.sources.loading")}
</p>
) : (
// The editor is always an available source (unconditionally prepended
// to availableSources), so the list is never empty — no "no sources"
// state exists.
// The backend always returns the editor as a virtual source, so the
// loaded list is never empty - no "no sources" state exists.
<div className="portal-policies__sources">
{availableSources.map((src) => (
// A selectable multi-line tile (icon + name + type + check).
@@ -0,0 +1,28 @@
import LocalIcon from "@app/components/shared/LocalIcon";
import type { FlowOutcomeKey } from "@portal/api/processorFlow";
import {
EDITOR_TYPE,
ICON_SIZE,
} from "@portal/components/processor-flow/flowTypes";
/** Real Material Symbols icon for a live source node, keyed off its `type`. */
export function SourceIcon({ type }: { type: string }) {
switch (type) {
case EDITOR_TYPE:
return <LocalIcon icon="edit-document" width={ICON_SIZE} />;
case "s3":
return <LocalIcon icon="cloud" width={ICON_SIZE} />;
case "folder":
return <LocalIcon icon="folder" width={ICON_SIZE} />;
default:
return <LocalIcon icon="database" width={ICON_SIZE} />;
}
}
/** Real Material Symbols icon for an audit outcome node. Literal icon names
* (not a ternary) so the icon extractor bundles them. */
export function OutcomeIcon({ outcome }: { outcome: FlowOutcomeKey }) {
if (outcome === "success")
return <LocalIcon icon="check-circle" width={ICON_SIZE} />;
return <LocalIcon icon="cancel" width={ICON_SIZE} />;
}
@@ -0,0 +1,59 @@
import { type RefObject } from "react";
import { useTranslation } from "react-i18next";
import { Button } from "@app/ui";
import type { FlowOutcome } from "@portal/api/processorFlow";
import { OutcomeIcon } from "@portal/components/processor-flow/FlowIcons";
interface FlowOutcomesProps {
outcomes: FlowOutcome[];
/** One ref slot per outcome, in order, for geometry measurement. */
outRefs: RefObject<(HTMLElement | null)[]>;
onOpen: () => void;
}
/** Right column: terminal audit outcomes (delivered / failed). */
export function FlowOutcomes({ outcomes, outRefs, onOpen }: FlowOutcomesProps) {
const { t } = useTranslation();
return (
<section
className="portal-pf__col portal-pf__col--outcomes"
aria-label={t("portal.processorFlow.outcomes.heading")}
>
<span className="portal-pf__col-head">
{t("portal.processorFlow.outcomes.heading")}
</span>
{outcomes.map((outcome, j) => (
<Button
key={outcome.key}
variant="quiet"
justify="start"
fullWidth
px="sm"
py="sm"
className={
"portal-pf__node portal-pf__node--outcome portal-pf__node--" +
outcome.key
}
onClick={onOpen}
ref={(el: HTMLButtonElement | null) => {
outRefs.current[j] = el;
}}
leftSection={
<span className="portal-pf__node-icon" aria-hidden>
<OutcomeIcon outcome={outcome.key} />
</span>
}
>
<span className="portal-pf__node-text">
<strong>{t(outcome.labelKey)}</strong>
<span>
{t("portal.processorFlow.outcomes.count", {
n: outcome.count24h,
})}
</span>
</span>
</Button>
))}
</section>
);
}
@@ -0,0 +1,80 @@
import { type RefObject } from "react";
import { useTranslation } from "react-i18next";
import { Button } from "@app/ui";
import LocalIcon from "@app/components/shared/LocalIcon";
import type { FlowPolicy } from "@portal/api/processorFlow";
import { ICON_SIZE } from "@portal/components/processor-flow/flowTypes";
interface FlowPoliciesProps {
policies: FlowPolicy[];
activeCount: number;
/** Ref for the core card (measured as the particle waist). */
coreRef: RefObject<HTMLDivElement | null>;
/** Per-policy lane-line refs, keyed by policy id, for particle threading. */
laneRefs: RefObject<Record<string, HTMLElement>>;
/** Deep-link into that policy's setup wizard. */
onSetup: (key: string) => void;
}
/** Centre column: the standing-policy catalogue — the particle "waist". */
export function FlowPolicies({
policies,
activeCount,
coreRef,
laneRefs,
onSetup,
}: FlowPoliciesProps) {
const { t } = useTranslation();
return (
<div className="portal-pf__policies" ref={coreRef}>
<div className="portal-pf__policies-head">
<span>{t("portal.processorFlow.policies.heading")}</span>
<span className="portal-pf__policies-active">
{t("portal.processorFlow.policies.activeCount", { n: activeCount })}
</span>
</div>
{policies.map((policy) => (
<div
key={policy.key}
className={"portal-pf__policy portal-pf__policy--" + policy.state}
>
<div
className="portal-pf__policy-line"
ref={(el: HTMLDivElement | null) => {
if (el) laneRefs.current[policy.key] = el;
else delete laneRefs.current[policy.key];
}}
>
<span className="portal-pf__policy-icon" aria-hidden>
<LocalIcon icon={policy.icon} width={ICON_SIZE} />
</span>
<span className="portal-pf__policy-label">
{t(policy.labelKey)}
</span>
{policy.state === "active" ? (
<span className="portal-pf__policy-count">
{t("portal.processorFlow.policies.count", {
n: policy.runs24h,
})}
</span>
) : policy.state === "off" ? (
<Button
size="sm"
py="xs"
variant="primary"
className="portal-pf__setup"
onClick={() => onSetup(policy.key)}
>
{t("portal.processorFlow.policies.setUp")}
</Button>
) : (
<span className="portal-pf__policy-soon">
{t("portal.processorFlow.policies.soon")}
</span>
)}
</div>
</div>
))}
</div>
);
}
@@ -0,0 +1,237 @@
import { type ReactNode } from "react";
import { useTranslation } from "react-i18next";
import { EmptyState } from "@app/ui";
import type {
FlowOutcome,
FlowOutcomeKey,
FlowPolicy,
FlowSource,
} from "@portal/api/processorFlow";
import {
EDITOR_TYPE,
OUTCOME_FILL,
} from "@portal/components/processor-flow/flowTypes";
interface FlowSankeyProps {
sources: FlowSource[];
outcomes: FlowOutcome[];
policies: FlowPolicy[];
}
/**
* Sankey lens: sources → policies waist → outcomes, ribbon width ∝ 24h volume.
* The waist splits into one segment per active policy. Shows a friendly empty
* state when nothing has flowed yet.
*/
export function FlowSankey({ sources, outcomes, policies }: FlowSankeyProps) {
const { t } = useTranslation();
const activePolicies = policies.filter((p) => p.state === "active");
const activeCount = activePolicies.length;
const flows = sources.filter((s) => s.docs24h > 0);
const srcSum = flows.reduce((sum, s) => sum + s.docs24h, 0);
if (!srcSum) {
return (
<div className="portal-pf__sankey-empty">
<EmptyState
size="compact"
title={t("portal.processorFlow.sankey.empty.title")}
description={t("portal.processorFlow.sankey.empty.description")}
/>
</div>
);
}
const SW = 720;
const SH = 220;
const padY = 22;
const xL = 180;
const xR = 560;
const xM = (xL + xR) / 2;
const barW = 9;
const midW = 11;
const gap = 12;
const H = SH - padY * 2;
const k = (H - (flows.length - 1) * gap) / srcSum;
const lt = flows.map((s) => Math.max(3, s.docs24h * k));
const midH = lt.reduce((a, b) => a + b, 0);
const y0L = padY + (H - (midH + (flows.length - 1) * gap)) / 2;
const midY = padY + (H - midH) / 2;
const outSum = outcomes.reduce((a, o) => a + o.count24h, 0);
const rawRt = outcomes.map((o) =>
outSum > 0
? Math.max(3, midH * (o.count24h / outSum))
: midH / outcomes.length,
);
const rtSum = rawRt.reduce((a, b) => a + b, 0);
const rt = rawRt.map((v) => (v * midH) / rtSum);
const y0R = padY + (H - (midH + (outcomes.length - 1) * gap)) / 2;
const outFill = (key: FlowOutcomeKey) => OUTCOME_FILL[key];
const srcFill = "var(--color-blue)";
const waistFill = "var(--color-text-4)";
const ribbon = (
x0: number,
t0: number,
b0: number,
x1: number,
t1: number,
b1: number,
fill: string,
key: string,
) => {
const mx = (x0 + x1) / 2;
return (
<path
key={key}
d={`M ${x0} ${t0} C ${mx} ${t0}, ${mx} ${t1}, ${x1} ${t1} L ${x1} ${b1} C ${mx} ${b1}, ${mx} ${b0}, ${x0} ${b0} Z`}
style={{ fill }}
opacity={0.3}
/>
);
};
const wires: ReactNode[] = [];
const bars: ReactNode[] = [];
const texts: ReactNode[] = [];
// Left stage: sources → waist.
let accL = y0L;
let accM = midY;
flows.forEach((s, i) => {
wires.push(
ribbon(
xL + barW,
accL,
accL + lt[i],
xM,
accM,
accM + lt[i],
srcFill,
"wl" + i,
),
);
bars.push(
<rect
key={"bl" + i}
x={xL}
y={accL}
width={barW}
height={lt[i]}
rx={2}
style={{ fill: srcFill }}
opacity={0.9}
/>,
);
const label =
s.type === EDITOR_TYPE
? t("portal.processorFlow.sources.editor")
: s.name;
texts.push(
<text
key={"tl" + i}
x={xL - 10}
y={accL + lt[i] / 2 + 4}
textAnchor="end"
className="portal-pf__sankey-label"
>
{label} · {s.docs24h}
</text>,
);
accL += lt[i] + gap;
accM += lt[i];
});
// Waist: one segment per active policy (sized by its 24h runs) so the centre
// reads as distinct policies rather than a single bar.
const segGap = 4;
const nSeg = Math.max(activePolicies.length, 1);
const segAvail = midH - (nSeg - 1) * segGap;
const polSum = activePolicies.reduce((a, p) => a + p.runs24h, 0);
const segPolicies = activePolicies.length ? activePolicies : [null];
let segY = midY;
segPolicies.forEach((p, i) => {
const wgt = p && polSum > 0 ? p.runs24h / polSum : 1 / nSeg;
const h = Math.max(2, segAvail * wgt);
bars.push(
<rect
key={"waist" + i}
x={xM}
y={segY}
width={midW}
height={h}
rx={2}
style={{ fill: waistFill }}
opacity={0.9}
/>,
);
segY += h + segGap;
});
texts.push(
<text
key="waist-cap"
x={xM + midW / 2}
y={midY - 8}
textAnchor="middle"
className="portal-pf__sankey-caption"
>
{t("portal.processorFlow.sankey.waist", { n: activeCount })}
</text>,
);
// Right stage: waist → outcomes.
let accWaist = midY;
let accR = y0R;
outcomes.forEach((o, j) => {
wires.push(
ribbon(
xM + midW,
accWaist,
accWaist + rt[j],
xR,
accR,
accR + rt[j],
outFill(o.key),
"wr" + j,
),
);
bars.push(
<rect
key={"br" + j}
x={xR}
y={accR}
width={barW}
height={rt[j]}
rx={2}
style={{ fill: outFill(o.key) }}
opacity={0.9}
/>,
);
texts.push(
<text
key={"tr" + j}
x={xR + barW + 10}
y={accR + rt[j] / 2 + 4}
textAnchor="start"
className="portal-pf__sankey-label"
>
{t(o.labelKey)} · {o.count24h}
</text>,
);
accWaist += rt[j];
accR += rt[j] + gap;
});
return (
<div className="portal-pf__sankey">
<svg viewBox={`0 0 ${SW} ${SH}`} width="100%" aria-hidden>
{wires}
{bars}
{texts}
</svg>
</div>
);
}
@@ -0,0 +1,94 @@
import { type RefObject } from "react";
import { useTranslation } from "react-i18next";
import { Button } from "@app/ui";
import LocalIcon from "@app/components/shared/LocalIcon";
import type {
FlowComingSoonSource,
FlowSource,
} from "@portal/api/processorFlow";
import {
EDITOR_TYPE,
ICON_SIZE,
} from "@portal/components/processor-flow/flowTypes";
import { SourceIcon } from "@portal/components/processor-flow/FlowIcons";
interface FlowSourcesProps {
sources: FlowSource[];
comingSoonSources: FlowComingSoonSource[];
/** One ref slot per live source, in order, for geometry measurement. */
srcRefs: RefObject<(HTMLElement | null)[]>;
onOpen: () => void;
}
/** Left column: live source cards (measured) + coming-soon connect cards. */
export function FlowSources({
sources,
comingSoonSources,
srcRefs,
onOpen,
}: FlowSourcesProps) {
const { t } = useTranslation();
return (
<section
className="portal-pf__col portal-pf__col--sources"
aria-label={t("portal.processorFlow.sources.heading")}
>
<span className="portal-pf__col-head">
{t("portal.processorFlow.sources.heading")}
</span>
{sources.map((source, i) => (
<Button
key={source.id}
variant="quiet"
justify="start"
fullWidth
px="sm"
py="sm"
className="portal-pf__node portal-pf__node--source"
onClick={onOpen}
ref={(el: HTMLButtonElement | null) => {
srcRefs.current[i] = el;
}}
leftSection={
<span className="portal-pf__node-icon" aria-hidden>
<SourceIcon type={source.type} />
</span>
}
>
<span className="portal-pf__node-text">
<strong>
{source.type === EDITOR_TYPE
? t("portal.processorFlow.sources.editor")
: source.name}
</strong>
<span>
{t("portal.processorFlow.sources.perDay", { n: source.docs24h })}
</span>
</span>
</Button>
))}
{comingSoonSources.map((cs) => (
<Button
key={cs.key}
variant="quiet"
justify="start"
fullWidth
px="sm"
py="sm"
className="portal-pf__node portal-pf__node--soon"
onClick={onOpen}
leftSection={
<span className="portal-pf__node-icon" aria-hidden>
<LocalIcon icon="add" width={ICON_SIZE} />
</span>
}
>
<span className="portal-pf__node-text">
<strong>{t(cs.labelKey)}</strong>
<span>{t("portal.processorFlow.sources.comingSoonTag")}</span>
</span>
</Button>
))}
</section>
);
}
@@ -0,0 +1,103 @@
import type { FlowOutcomeKey } from "@portal/api/processorFlow";
/** Which lens the visualiser is showing. */
export type Lens = "flow" | "sankey";
/**
* DEV ONLY: force the flow animation on even when nothing is set up / no
* activity has taken place, using synthetic rates. Off by design — an idle
* machine shows no animated dots; flip to true only to preview the motion
* while iterating on an empty workspace.
*/
export const DEV_KEEP_FLOWING = false;
export const EDITOR_TYPE = "editor";
/** Emission tuning: particles/sec for a source ≈ rate / 86400 × SPEED. */
export const SPEED = 300;
/** Synthetic per-source rate used only while DEV_KEEP_FLOWING forces the flow. */
export const DEV_SYNTH_RATE = 320;
/** Hard cap on live particles (matches the reference). */
export const MAX_PARTICLES = 36;
/** No two dots leave the same source within this window (ms). */
export const MIN_EMIT_GAP = 200;
export const ICON_SIZE = "1.125rem";
/** SVG `fill` (a CSS property, so var() resolves per-theme) for each outcome. */
export const OUTCOME_FILL: Record<FlowOutcomeKey, string> = {
success: "var(--color-green)",
failed: "var(--color-red)",
};
/* ── Measured geometry ──────────────────────────────────────────────────── */
export interface Rect {
l: number;
r: number;
t: number;
b: number;
cy: number;
}
export interface Lane {
key: string;
cy: number;
el: HTMLElement;
}
export interface Geo {
w: number;
h: number;
srcs: (Rect | undefined)[];
outs: (Rect | undefined)[];
core: Rect | null;
lanes: Lane[];
}
export interface Point {
x: number;
y: number;
}
export interface Particle {
el: SVGCircleElement;
src: number;
out: number;
lane: string | null;
phase: 0 | 1 | 2;
t: number;
d0: number;
d1: number;
d2: number;
pulsed: boolean;
}
/** Cubic bézier point at t. */
export function cbez(a: Point, b: Point, c: Point, d: Point, t: number): Point {
const m = 1 - t;
return {
x:
m * m * m * a.x +
3 * m * m * t * b.x +
3 * m * t * t * c.x +
t * t * t * d.x,
y:
m * m * m * a.y +
3 * m * m * t * b.y +
3 * m * t * t * c.y +
t * t * t * d.y,
};
}
/** Smoothstep easing used for the in-card lane glide. */
export function smooth(t: number): number {
return t * t * (3 - 2 * t);
}
/** Where each source's wire enters the core (spread across its height). */
export function coreEntryY(g: Geo, i: number): number {
if (!g.core) return 0;
const n = Math.max(g.srcs.length, 2);
return g.core.t + 34 + (g.core.b - g.core.t - 68) * (i / (n - 1));
}
@@ -0,0 +1,123 @@
import {
useEffect,
useLayoutEffect,
useRef,
useState,
type ReactNode,
} from "react";
import {
coreEntryY,
type Geo,
type Rect,
} from "@portal/components/processor-flow/flowTypes";
/**
* Owns the measured-geometry seam for the flow visualiser: refs for the source,
* outcome and core cards (plus the per-policy lane lines), a `measure()` that
* projects their edges into a wrapper-relative {@link Geo}, and the SVG wires
* drawn between them. The particle loop reads the same `geoRef` live.
*
* Callers spread the returned refs onto the cards and render `wires` inside the
* underlay `<svg>`; geometry re-measures on every layout + on resize, and the
* wires re-render only when the measured signature actually changes.
*/
export function useFlowGeometry() {
const wrapRef = useRef<HTMLDivElement>(null);
const srcRefs = useRef<(HTMLElement | null)[]>([]);
const outRefs = useRef<(HTMLElement | null)[]>([]);
const coreRef = useRef<HTMLDivElement>(null);
const laneRefs = useRef<Record<string, HTMLElement>>({});
const geoRef = useRef<Geo | null>(null);
const geoSigRef = useRef("");
const [, setGeoTick] = useState(0);
const measure = () => {
const w = wrapRef.current;
if (!w) return;
const wr = w.getBoundingClientRect();
if (!wr.width) return;
const rel = (r: DOMRect): Rect => ({
l: r.left - wr.left,
r: r.right - wr.left,
t: r.top - wr.top,
b: r.bottom - wr.top,
cy: r.top - wr.top + r.height / 2,
});
const g: Geo = {
w: wr.width,
h: wr.height,
srcs: [],
outs: [],
core: null,
lanes: [],
};
srcRefs.current.forEach((el, i) => {
if (el) g.srcs[i] = rel(el.getBoundingClientRect());
});
outRefs.current.forEach((el, j) => {
if (el) g.outs[j] = rel(el.getBoundingClientRect());
});
if (coreRef.current) g.core = rel(coreRef.current.getBoundingClientRect());
Object.entries(laneRefs.current).forEach(([key, el]) => {
if (el && el.isConnected)
g.lanes.push({ key, cy: rel(el.getBoundingClientRect()).cy, el });
});
geoRef.current = g;
let cySum = 0;
g.srcs.forEach((s) => s && (cySum += s.cy));
g.outs.forEach((o) => o && (cySum += o.cy));
const sig = [
Math.round(g.w),
Math.round(g.h),
g.srcs.length,
g.outs.length,
g.core ? Math.round(g.core.t) + ":" + Math.round(g.core.b) : 0,
Math.round(cySum),
].join(":");
if (sig !== geoSigRef.current) {
geoSigRef.current = sig;
setGeoTick((n) => n + 1);
}
};
useLayoutEffect(measure);
useEffect(() => {
const onResize = () => measure();
window.addEventListener("resize", onResize);
return () => window.removeEventListener("resize", onResize);
}, []);
// Wires (SVG underlay); recomputed whenever the geometry signature changes.
const g = geoRef.current;
let wires: ReactNode = null;
if (g && g.core) {
const core = g.core;
const paths: ReactNode[] = [];
g.srcs.forEach((s, i) => {
if (!s) return;
const ty = coreEntryY(g, i);
paths.push(
<path
key={"ws" + i}
className="portal-pf__wire-path"
d={`M ${s.r} ${s.cy} C ${s.r + 44} ${s.cy}, ${core.l - 44} ${ty}, ${core.l} ${ty}`}
/>,
);
});
g.outs.forEach((o, j) => {
if (!o) return;
paths.push(
<path
key={"wo" + j}
className="portal-pf__wire-path"
d={`M ${core.r} ${o.cy} C ${core.r + 44} ${o.cy}, ${o.l - 44} ${o.cy}, ${o.l} ${o.cy}`}
/>,
);
});
wires = paths;
}
return { wrapRef, srcRefs, outRefs, coreRef, laneRefs, geoRef, wires };
}
@@ -0,0 +1,261 @@
import { useEffect, useRef, type RefObject } from "react";
import type { FlowOutcomeKey } from "@portal/api/processorFlow";
import {
cbez,
coreEntryY,
smooth,
MAX_PARTICLES,
MIN_EMIT_GAP,
OUTCOME_FILL,
SPEED,
type Geo,
type Lens,
type Particle,
type Point,
} from "@portal/components/processor-flow/flowTypes";
interface FlowParticlesOptions {
geoRef: RefObject<Geo | null>;
animate: boolean;
lens: Lens;
/** Per-source emission rate (docs/24h, or synthetic while dev-forcing). */
rates: number[];
/** Outcome share for the weighted round-robin destination picker. */
weights: number[];
/** Policy lane keys a dot may thread through the core. */
laneKeys: string[];
/** Outcome keys, index-aligned with `weights`, for recolouring on arrival. */
outcomeKeys: FlowOutcomeKey[];
}
/**
* Drives the rAF particle loop: emits dots per source on a jittered schedule
* (min-gap floored), routes each to an outcome via a weighted round-robin so
* the split matches the counts, threads it through a policy lane (blinking that
* row's LED), and recolours it to the outcome on arrival. Reads geometry live
* from `geoRef`, so it tracks card movement without restarting.
*
* Returns the `<g>` ref the caller mounts inside the particle overlay `<svg>`.
* The loop only runs on the flow lens, when `animate` is set, and outside
* reduced-motion; browsers pause rAF for hidden tabs (desirable).
*/
export function useFlowParticles({
geoRef,
animate,
lens,
rates,
weights,
laneKeys,
outcomeKeys,
}: FlowParticlesOptions): RefObject<SVGGElement | null> {
const pGroupRef = useRef<SVGGElement>(null);
// Restart the loop only when the meaningful inputs change.
const flowSig = [
animate,
lens,
rates.join(","),
laneKeys.join(","),
weights.map((w) => w.toFixed(3)).join(","),
outcomeKeys.join(","),
].join("|");
useEffect(() => {
if (!animate || lens !== "flow") return;
const reduced =
typeof window !== "undefined" &&
window.matchMedia &&
window.matchMedia("(prefers-reduced-motion: reduce)").matches;
if (reduced) return;
const pg = pGroupRef.current;
if (!pg) return;
const NS = "http://www.w3.org/2000/svg";
const particles: Particle[] = [];
let last = performance.now();
let raf = 0;
const glowTimers: Record<string, ReturnType<typeof setTimeout>> = {};
// Per-source emission schedule. Mean interval keeps each source's share of
// the flow proportional to its rate; the scheduled model (vs. a steady
// accumulator) is what lets us jitter departures and floor the gap.
const meanInterval = rates.map((r) => {
const perSec = (r / 86400) * SPEED;
return perSec > 0 ? 1000 / perSec : Infinity;
});
// Stagger the first emission so sources don't all fire together at t=0.
const nextEmit = meanInterval.map((mi) =>
Number.isFinite(mi) ? last + Math.random() * mi : Infinity,
);
// Random departure within [0.5×, 1.5×] the mean, but never closer than the
// minimum gap — a random flow with no two dots out at once per source.
const scheduleNext = (i: number, now: number): number =>
now + Math.max(MIN_EMIT_GAP, meanInterval[i] * (0.5 + Math.random()));
// Weighted round-robin so the outcome split visibly matches the counts
// (e.g. 3 failed / 30 delivered → ~1 in 11 dots to Failed), interleaved
// rather than clustered like independent random draws.
const outAcc = weights.map(() => 0);
const pickOut = (): number => {
if (!weights.length) return 0;
for (let j = 0; j < weights.length; j++) outAcc[j] += weights[j];
let best = 0;
for (let j = 1; j < weights.length; j++) {
if (outAcc[j] > outAcc[best]) best = j;
}
outAcc[best] -= 1;
return best;
};
const pickLane = (): string | null => {
if (!laneKeys.length) return null;
return laneKeys[Math.floor(Math.random() * laneKeys.length)];
};
const laneY = (g: Geo, key: string | null): number | null => {
if (!key) return null;
const l = g.lanes.find((x) => x.key === key);
return l ? l.cy : null;
};
// Blink the row's leading LED (its icon) for 150ms as a particle threads it.
const pulseLane = (g: Geo, key: string | null) => {
if (!key) return;
const lane = g.lanes.find((x) => x.key === key);
const led = lane?.el.firstElementChild;
if (!led || !led.classList.contains("portal-pf__policy-icon")) return;
led.classList.add("is-pulse");
if (glowTimers[key]) clearTimeout(glowTimers[key]);
glowTimers[key] = setTimeout(() => led.classList.remove("is-pulse"), 150);
};
const frame = (now: number) => {
const g = geoRef.current;
const dt = Math.min(now - last, 200);
last = now;
if (g && g.core) {
// At most one dot per source per frame, once its scheduled (jittered)
// departure time is reached — guarantees the per-source minimum gap.
for (let i = 0; i < meanInterval.length; i++) {
if (!Number.isFinite(meanInterval[i]) || !g.srcs[i]) continue;
if (now >= nextEmit[i] && particles.length < MAX_PARTICLES) {
const c = document.createElementNS(
NS,
"circle",
) as SVGCircleElement;
c.setAttribute("r", "2.5");
c.setAttribute("opacity", "0.75");
c.style.fill = "var(--color-blue)";
pg.appendChild(c);
particles.push({
el: c,
src: i,
out: pickOut(),
lane: pickLane(),
phase: 0,
t: 0,
d0: 900 + Math.random() * 300,
d1: 760,
d2: 780 + Math.random() * 200,
pulsed: false,
});
nextEmit[i] = scheduleNext(i, now);
}
}
for (let k = particles.length - 1; k >= 0; k--) {
const p = particles[k];
p.t += dt;
const s = g.srcs[p.src];
if (!s) {
p.el.remove();
particles.splice(k, 1);
continue;
}
let pos: Point;
if (p.phase === 0) {
const ey = coreEntryY(g, p.src);
const f0 = Math.min(1, p.t / p.d0);
pos = cbez(
{ x: s.r, y: s.cy },
{ x: s.r + 44, y: s.cy },
{ x: g.core.l - 44, y: ey },
{ x: g.core.l, y: ey },
f0,
);
if (f0 >= 1) {
p.phase = 1;
p.t = 0;
p.pulsed = false;
p.el.setAttribute("r", "2");
p.el.setAttribute("opacity", "0.45");
}
} else if (p.phase === 1) {
const o1 = g.outs[p.out];
if (!o1) {
p.el.remove();
particles.splice(k, 1);
continue;
}
const f1 = Math.min(1, p.t / p.d1);
const entY = coreEntryY(g, p.src);
const exitY = o1.cy;
const ly1 = laneY(g, p.lane);
let yy: number;
if (ly1 == null) {
yy = entY + (exitY - entY) * f1;
} else if (f1 < 0.25) {
yy = entY + (ly1 - entY) * smooth(f1 / 0.25);
} else if (f1 < 0.75) {
yy = ly1;
if (!p.pulsed) {
p.pulsed = true;
pulseLane(g, p.lane);
}
} else {
yy = ly1 + (exitY - ly1) * smooth((f1 - 0.75) / 0.25);
}
pos = { x: g.core.l + (g.core.r - g.core.l) * f1, y: yy };
if (f1 >= 1) {
p.phase = 2;
p.t = 0;
p.el.style.fill =
OUTCOME_FILL[outcomeKeys[p.out]] ?? "var(--color-blue)";
p.el.setAttribute("r", "2.5");
p.el.setAttribute("opacity", "0.75");
}
} else {
const o2 = g.outs[p.out];
if (!o2) {
p.el.remove();
particles.splice(k, 1);
continue;
}
const f2 = Math.min(1, p.t / p.d2);
pos = cbez(
{ x: g.core.r, y: o2.cy },
{ x: g.core.r + 44, y: o2.cy },
{ x: o2.l - 44, y: o2.cy },
{ x: o2.l, y: o2.cy },
f2,
);
if (f2 >= 1) {
p.el.remove();
particles.splice(k, 1);
continue;
}
}
p.el.setAttribute("cx", String(pos.x));
p.el.setAttribute("cy", String(pos.y));
}
}
raf = requestAnimationFrame(frame);
};
raf = requestAnimationFrame(frame);
return () => {
cancelAnimationFrame(raf);
Object.values(glowTimers).forEach(clearTimeout);
while (pg.firstChild) pg.removeChild(pg.firstChild);
};
}, [flowSig, geoRef]);
return pGroupRef;
}
@@ -75,7 +75,7 @@ export function DocumentLedger({
className="portal-proc__stage-label"
data-current={cur || undefined}
>
{group.label}
{t(group.label)}
</span>
{blurb && (
<span className="portal-proc__stage-hint">
@@ -1,16 +0,0 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { ConnectWizard } from "@portal/components/sources/ConnectWizard";
const meta: Meta<typeof ConnectWizard> = {
title: "Portal/Sources/ConnectWizard",
component: ConnectWizard,
parameters: { layout: "fullscreen" },
args: { open: true, onClose: () => {}, onCreated: () => {} },
};
export default meta;
type Story = StoryObj<typeof ConnectWizard>;
/** Opens on the type-picker step; Continue/Back walk through the three steps. */
export const Open: Story = {};
export const Closed: Story = { args: { open: false } };
@@ -1,192 +0,0 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { HttpError } from "@portal/api/http";
import { ConnectWizard } from "@portal/components/sources/ConnectWizard";
function renderWithMantine(ui: React.ReactElement) {
return render(<MantineProvider>{ui}</MantineProvider>);
}
// Deterministic i18n: keys come back verbatim so the test never waits on the
// async TOML backend.
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => key,
i18n: { changeLanguage: vi.fn() },
}),
}));
// Mock the service layer so no real fetch is issued.
const createSource = vi.fn();
vi.mock("@portal/api/sources", () => ({
createSource: (...args: unknown[]) => createSource(...args),
}));
/** Step through the folder-source flow: choose type -> configure -> review. */
function stepToReview() {
// Step 0: folder is the default-selected type. Continue.
fireEvent.click(screen.getByText("portal.sources.wizard.continue"));
// Step 1: fill the required name + directory fields.
const inputs = screen.getAllByRole("textbox");
// First textbox is the name field, second is the folder's directory field.
fireEvent.change(inputs[0], { target: { value: "Claims intake" } });
fireEvent.change(inputs[1], { target: { value: "/data/incoming" } });
fireEvent.click(screen.getByText("portal.sources.wizard.continue"));
}
describe("ConnectWizard", () => {
beforeEach(() => {
createSource.mockReset();
});
it("creates a folder source with the configured options", async () => {
createSource.mockResolvedValue({ id: "src-1" });
const onCreated = vi.fn();
const onClose = vi.fn();
renderWithMantine(
<ConnectWizard open onClose={onClose} onCreated={onCreated} />,
);
stepToReview();
// Step 2: submit via the final "connect source" action.
fireEvent.click(screen.getByText("portal.sources.actions.connectSource"));
await waitFor(() => {
expect(createSource).toHaveBeenCalledTimes(1);
});
expect(createSource).toHaveBeenCalledWith({
name: "Claims intake",
type: "folder",
options: {
directory: "/data/incoming",
mode: "consume",
recursive: "false",
identity: "stat",
},
enabled: true,
});
await waitFor(() => {
expect(onCreated).toHaveBeenCalledTimes(1);
});
});
it("creates an s3 source with a masked secret in review", async () => {
createSource.mockResolvedValue({ id: "src-2" });
renderWithMantine(
<ConnectWizard open onClose={vi.fn()} onCreated={vi.fn()} />,
);
// Step 0: pick the S3 type card.
fireEvent.click(screen.getByText("portal.sources.types.s3.label"));
fireEvent.click(screen.getByText("portal.sources.wizard.continue"));
// Step 1: name, bucket, credentials. The secret renders as a password
// input, so it is not part of the textbox roles.
const inputs = screen.getAllByRole("textbox") as HTMLInputElement[];
fireEvent.change(inputs[0], { target: { value: "Claims bucket" } });
fireEvent.change(inputs[1], { target: { value: "claims-inbox" } });
fireEvent.change(inputs[4], { target: { value: "AKIAEXAMPLE" } });
const secret = document.querySelector(
'input[type="password"]',
) as HTMLInputElement;
fireEvent.change(secret, { target: { value: "shh-secret" } });
fireEvent.click(screen.getByText("portal.sources.wizard.continue"));
// Step 2: the secret is masked in review, never echoed.
expect(screen.queryByText("shh-secret")).not.toBeInTheDocument();
expect(screen.getByText("********")).toBeInTheDocument();
fireEvent.click(screen.getByText("portal.sources.actions.connectSource"));
await waitFor(() => {
expect(createSource).toHaveBeenCalledTimes(1);
});
expect(createSource).toHaveBeenCalledWith({
name: "Claims bucket",
type: "s3",
options: {
bucket: "claims-inbox",
region: "us-east-1",
prefix: "",
accessKeyId: "AKIAEXAMPLE",
secretAccessKey: "shh-secret",
endpoint: "",
mode: "consume",
},
enabled: true,
});
});
it("edits an existing source: prefilled, skips type, submits with its id", async () => {
createSource.mockResolvedValue({ id: "s1" });
const onCreated = vi.fn();
renderWithMantine(
<ConnectWizard
open
source={{
id: "s1",
name: "James",
type: "folder",
options: { directory: "'/data/in'", mode: "consume" },
enabled: true,
}}
onClose={vi.fn()}
onCreated={onCreated}
/>,
);
// Edit opens on the configure step with name + directory prefilled.
const inputs = screen.getAllByRole("textbox") as HTMLInputElement[];
expect(inputs[0].value).toBe("James");
expect(inputs[1].value).toBe("'/data/in'");
// Fix the stray quotes, continue to review, save.
fireEvent.change(inputs[1], { target: { value: "/data/in" } });
fireEvent.click(screen.getByText("portal.sources.wizard.continue"));
fireEvent.click(screen.getByText("portal.sources.wizard.save"));
await waitFor(() => {
expect(createSource).toHaveBeenCalledTimes(1);
});
// Options absent from the stored source are submitted at their defaults.
expect(createSource).toHaveBeenCalledWith({
id: "s1",
name: "James",
type: "folder",
options: {
directory: "/data/in",
mode: "consume",
recursive: "false",
identity: "stat",
},
enabled: true,
});
await waitFor(() => {
expect(onCreated).toHaveBeenCalledTimes(1);
});
});
it("renders the inline error message when create fails", async () => {
createSource.mockRejectedValue(
new HttpError(400, "Bad Request", {
detail: "Directory is not readable",
}),
);
renderWithMantine(
<ConnectWizard open onClose={vi.fn()} onCreated={vi.fn()} />,
);
stepToReview();
fireEvent.click(screen.getByText("portal.sources.actions.connectSource"));
expect(
await screen.findByText("Directory is not readable"),
).toBeInTheDocument();
});
});
@@ -1,298 +0,0 @@
import { useEffect, useState } from "react";
import { useTranslation } from "react-i18next";
import {
Banner,
Button,
FormField,
Input,
Modal,
Select,
StatTile,
} from "@app/ui";
import { errorMessage } from "@portal/api/http";
import { createSource, type Source } from "@portal/api/sources";
import { creatableSourceTypes } from "@portal/components/sources/creatableSourceTypes";
import {
defaultOptions,
sourceTypeMeta,
type CreatableSourceType,
} from "@portal/components/sources/sourceTypes";
import "@portal/views/Sources.css";
const OFFERED_TYPES = creatableSourceTypes();
const DEFAULT_TYPE = OFFERED_TYPES[0];
/** Wizard steps. Editing skips type selection (the type is fixed once created). */
type StepId = "type" | "configure" | "review";
const CREATE_STEPS: StepId[] = ["type", "configure", "review"];
const EDIT_STEPS: StepId[] = ["configure", "review"];
interface ConnectWizardProps {
open: boolean;
onClose: () => void;
/** Called after a source is created or updated so the page can refetch. */
onCreated: () => void;
/** When set, the wizard edits this existing source instead of creating one. */
source?: Source;
}
/** The creatable-type metadata for a source's stored type, falling back to the first offered. */
function typeFor(type: string | undefined): CreatableSourceType {
return OFFERED_TYPES.find((t) => t.type === type) ?? DEFAULT_TYPE;
}
/** Source options coerced to strings for the form, defaulted from the type's fields. */
function optionsFor(
type: CreatableSourceType,
options: Record<string, unknown> | undefined,
): Record<string, string> {
const out = defaultOptions(type);
for (const [key, value] of Object.entries(options ?? {})) {
out[key] = value == null ? "" : String(value);
}
return out;
}
/**
* Guided flow for connecting a source (choose type -> configure -> review) or
* editing an existing one (configure -> review, type fixed). On submit a blank
* id creates and a set id updates, matching the backend's POST contract.
*/
export function ConnectWizard({
open,
onClose,
onCreated,
source,
}: ConnectWizardProps) {
const { t } = useTranslation();
const isEdit = source !== undefined;
const steps = isEdit ? EDIT_STEPS : CREATE_STEPS;
const [stepIndex, setStepIndex] = useState(0);
const [type, setType] = useState<CreatableSourceType>(() =>
typeFor(source?.type),
);
const [name, setName] = useState(source?.name ?? "");
const [options, setOptions] = useState<Record<string, string>>(() =>
optionsFor(typeFor(source?.type), source?.options),
);
const [submitting, setSubmitting] = useState(false);
const [error, setError] = useState<string | null>(null);
// Re-seed the form whenever the wizard opens (or its target source changes) so
// editing prefills the current config and a reopened create starts clean.
useEffect(() => {
if (!open) return;
const ct = typeFor(source?.type);
setStepIndex(0);
setType(ct);
setName(source?.name ?? "");
setOptions(optionsFor(ct, source?.options));
setSubmitting(false);
setError(null);
}, [open, source]);
const stepId = steps[stepIndex];
const isLast = stepIndex === steps.length - 1;
function chooseType(next: CreatableSourceType) {
setType(next);
setOptions(defaultOptions(next));
}
const requiredFilled = type.fields.every(
(f) => !f.required || (options[f.key] ?? "").trim() !== "",
);
const canContinue =
stepId === "configure" ? name.trim() !== "" && requiredFilled : true;
async function advance() {
if (!isLast) {
setStepIndex((i) => i + 1);
return;
}
setSubmitting(true);
setError(null);
try {
const fields = {
name: name.trim(),
type: type.type,
options,
enabled: source?.enabled ?? true,
};
await createSource(isEdit ? { ...fields, id: source.id } : fields);
onCreated();
onClose();
} catch (e) {
setError(errorMessage(e));
} finally {
setSubmitting(false);
}
}
const stepLabels: Record<StepId, string> = {
type: t("portal.sources.wizard.steps.chooseType"),
configure: t("portal.sources.wizard.steps.configure"),
review: t("portal.sources.wizard.steps.review"),
};
return (
<Modal
open={open}
onClose={onClose}
width="lg"
title={
isEdit
? t("portal.sources.wizard.editTitle")
: t("portal.sources.wizard.title")
}
subtitle={t("portal.sources.wizard.subtitle", {
current: stepIndex + 1,
total: steps.length,
label: stepLabels[stepId],
})}
footer={
<div className="portal-sources__wizard-footer">
<Button
variant="tertiary"
size="sm"
disabled={submitting}
onClick={() =>
stepIndex === 0 ? onClose() : setStepIndex((i) => i - 1)
}
>
{stepIndex === 0
? t("portal.sources.wizard.cancel")
: t("portal.sources.wizard.back")}
</Button>
<Button
size="sm"
onClick={advance}
loading={submitting}
disabled={!canContinue}
rightSection={!isLast ? <span aria-hidden>→</span> : undefined}
>
{!isLast
? t("portal.sources.wizard.continue")
: isEdit
? t("portal.sources.wizard.save")
: t("portal.sources.actions.connectSource")}
</Button>
</div>
}
>
<ol className="portal-sources__steps" aria-hidden>
{steps.map((id, i) => (
<li
key={id}
className={
"portal-sources__step" +
(i === stepIndex ? " is-active" : i < stepIndex ? " is-done" : "")
}
>
<span className="portal-sources__step-mark">
{i < stepIndex ? "✓" : i + 1}
</span>
{stepLabels[id]}
</li>
))}
</ol>
{stepId === "type" && (
<div className="portal-sources__type-grid">
{OFFERED_TYPES.map((ct) => (
<Button
key={ct.type}
variant="tertiary"
className={
"portal-sources__type-card" +
(type.type === ct.type ? " is-selected" : "")
}
onClick={() => chooseType(ct)}
>
<span className="portal-sources__type-icon" aria-hidden>
{sourceTypeMeta(ct.type).icon}
</span>
<span className="portal-sources__type-name">
{t(ct.labelKey)}
</span>
</Button>
))}
</div>
)}
{stepId === "configure" && (
<div className="portal-sources__wizard-body">
<FormField label={t("portal.sources.wizard.name")} required>
<Input
value={name}
placeholder={t("portal.sources.wizard.namePlaceholder")}
onChange={(e) => setName(e.target.value)}
/>
</FormField>
{type.fields.map((field) => (
<FormField
key={field.key}
label={t(field.labelKey)}
helperText={
field.helperTextKey ? t(field.helperTextKey) : undefined
}
required={field.required}
>
{field.control === "select" ? (
<Select
value={options[field.key] ?? ""}
options={(field.options ?? []).map((o) => ({
value: o.value,
label: t(o.labelKey),
}))}
onChange={(value) =>
setOptions((o) => ({ ...o, [field.key]: value ?? "" }))
}
/>
) : (
<Input
type={field.control === "password" ? "password" : undefined}
value={options[field.key] ?? ""}
placeholder={
field.placeholderKey ? t(field.placeholderKey) : undefined
}
onChange={(e) =>
setOptions((o) => ({ ...o, [field.key]: e.target.value }))
}
/>
)}
</FormField>
))}
</div>
)}
{stepId === "review" && (
<div className="portal-sources__wizard-body">
<div className="portal-sources__stat-grid">
<StatTile
label={t("portal.sources.wizard.name")}
value={name || "—"}
/>
<StatTile
label={t("portal.sources.wizard.type")}
value={t(type.labelKey)}
/>
{type.fields.map((field) => (
<StatTile
key={field.key}
label={t(field.labelKey)}
value={
field.control === "password" && options[field.key]
? "********"
: options[field.key] || "—"
}
/>
))}
</div>
{error && <Banner tone="danger" description={error} />}
</div>
)}
</Modal>
);
}
@@ -0,0 +1,82 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
fireEvent,
render as baseRender,
screen,
waitFor,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { HttpError } from "@portal/api/http";
import { ConnectionsTab } from "@portal/components/sources/ConnectionsTab";
import type { IntegrationConfig } from "@portal/api/integrations";
const render = (ui: Parameters<typeof baseRender>[0]) =>
baseRender(ui, { wrapper: MantineProvider });
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => key,
i18n: { changeLanguage: vi.fn() },
}),
}));
const fetchS3Connections = vi.fn();
const deleteIntegration = vi.fn();
vi.mock("@portal/api/integrations", () => ({
fetchS3Connections: () => fetchS3Connections(),
deleteIntegration: (id: number) => deleteIntegration(id),
createIntegration: vi.fn(),
updateIntegration: vi.fn(),
}));
const CONNECTION = {
id: 5,
integrationType: "S3",
name: "Claims bucket",
config: { bucket: "inbox", region: "us-east-1" },
canManage: true,
} as unknown as IntegrationConfig;
describe("ConnectionsTab", () => {
beforeEach(() => {
fetchS3Connections.mockReset();
deleteIntegration.mockReset();
deleteIntegration.mockResolvedValue(undefined);
});
it("shows the empty state when there are no connections", async () => {
fetchS3Connections.mockResolvedValue([]);
render(<ConnectionsTab />);
expect(
await screen.findByText("portal.connections.empty.title"),
).toBeInTheDocument();
});
it("lists connections and deletes one", async () => {
fetchS3Connections.mockResolvedValueOnce([CONNECTION]);
fetchS3Connections.mockResolvedValueOnce([]);
render(<ConnectionsTab />);
expect(await screen.findByText("Claims bucket")).toBeInTheDocument();
fireEvent.click(screen.getByText("portal.connections.delete"));
await waitFor(() => expect(deleteIntegration).toHaveBeenCalledWith(5));
});
it("surfaces the 409 when deleting a connection still in use", async () => {
fetchS3Connections.mockResolvedValue([CONNECTION]);
deleteIntegration.mockRejectedValue(
new HttpError(409, "Conflict", {
detail: "Integration is in use by: source 'Claims intake'",
}),
);
render(<ConnectionsTab />);
await screen.findByText("Claims bucket");
fireEvent.click(screen.getByText("portal.connections.delete"));
expect(
await screen.findByText(
"Integration is in use by: source 'Claims intake'",
),
).toBeInTheDocument();
});
});
@@ -0,0 +1,186 @@
import { useCallback, useEffect, useMemo, useState } from "react";
import { useTranslation } from "react-i18next";
import AddRoundedIcon from "@mui/icons-material/AddRounded";
import {
Banner,
Button,
EmptyState,
Skeleton,
Table,
type TableColumn,
} from "@app/ui";
import { errorMessage } from "@portal/api/http";
import {
deleteIntegration,
fetchS3Connections,
type IntegrationConfig,
} from "@portal/api/integrations";
import { SourcesIcon } from "@portal/components/icons";
import { S3ConnectionModal } from "@portal/components/sources/S3ConnectionModal";
/**
* The Connections tab of the Sources page: stored S3 connections that sources
* and pipeline outputs reference by id. Create/edit go through the shared
* {@link S3ConnectionModal}; deleting one the backend still references returns a
* 409, surfaced inline.
*/
export function ConnectionsTab() {
const { t } = useTranslation();
const [connections, setConnections] = useState<IntegrationConfig[] | null>(
null,
);
const [modalOpen, setModalOpen] = useState(false);
const [editing, setEditing] = useState<IntegrationConfig | null>(null);
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const refresh = useCallback(async () => {
try {
setConnections(await fetchS3Connections());
} catch (e) {
setError(errorMessage(e));
}
}, []);
useEffect(() => {
void refresh();
}, [refresh]);
function openCreate() {
setEditing(null);
setModalOpen(true);
}
function openEdit(connection: IntegrationConfig) {
setEditing(connection);
setModalOpen(true);
}
async function remove(connection: IntegrationConfig) {
if (busy) return;
setBusy(true);
setError(null);
try {
await deleteIntegration(connection.id);
await refresh();
} catch (e) {
setError(errorMessage(e));
} finally {
setBusy(false);
}
}
const columns = useMemo<TableColumn<IntegrationConfig>[]>(
() => [
{
key: "name",
header: t("portal.connections.table.name"),
render: (c) => <strong>{c.name}</strong>,
},
{
key: "bucket",
header: t("portal.connections.table.bucket"),
render: (c) => (
<span className="portal-sources__connections-bucket">
{String(c.config?.bucket ?? "")}
</span>
),
},
{
key: "region",
header: t("portal.connections.table.region"),
render: (c) => String(c.config?.region ?? ""),
},
{
key: "actions",
header: "",
align: "right",
render: (c) =>
c.canManage ? (
<span className="portal-sources__connections-actions">
<Button
variant="tertiary"
size="sm"
disabled={busy}
onClick={() => openEdit(c)}
>
{t("portal.connections.edit")}
</Button>
<Button
variant="tertiary"
size="sm"
accent="danger"
disabled={busy}
onClick={() => void remove(c)}
>
{t("portal.connections.delete")}
</Button>
</span>
) : null,
},
],
// remove/openEdit are stable enough for this admin surface; busy gates them.
[t, busy],
);
const isLoading = connections === null;
const isEmpty = connections !== null && connections.length === 0;
return (
<section className="portal-sources__connections">
<div className="portal-sources__connections-head">
<p className="portal-sources__connections-sub">
{t("portal.connections.subtitle")}
</p>
<Button
onClick={openCreate}
leftSection={<AddRoundedIcon style={{ fontSize: "1.125rem" }} />}
>
{t("portal.connections.actions.new")}
</Button>
</div>
{error && <Banner tone="danger" description={error} />}
{isLoading && (
<div className="portal-sources__table-skeleton" aria-hidden>
{Array.from({ length: 3 }).map((_, i) => (
<Skeleton key={i} height="3rem" />
))}
</div>
)}
{isEmpty && (
<EmptyState
icon={<SourcesIcon size={28} />}
title={t("portal.connections.empty.title")}
description={t("portal.connections.empty.description")}
actions={
<Button
onClick={openCreate}
leftSection={<AddRoundedIcon style={{ fontSize: "1.125rem" }} />}
>
{t("portal.connections.actions.new")}
</Button>
}
/>
)}
{connections !== null && connections.length > 0 && (
<Table<IntegrationConfig>
className="portal-sources__connections-table"
columns={columns}
rows={connections}
rowKey={(c) => String(c.id)}
/>
)}
<S3ConnectionModal
open={modalOpen}
connection={editing}
onClose={() => setModalOpen(false)}
onSaved={() => void refresh()}
/>
</section>
);
}
@@ -0,0 +1,116 @@
import { useTranslation } from "react-i18next";
import { FormField, Input } from "@app/ui";
/**
* The connection-level S3 fields (per-use settings like prefix/mode live on the
* source or output referencing the connection). Secrets are write-only: when
* editing, the backend returns them masked and keeps the stored value if the
* mask is sent back unchanged.
*/
export interface S3ConnectionFormValues {
name: string;
bucket: string;
region: string;
endpoint: string;
accessKeyId: string;
secretAccessKey: string;
}
export const EMPTY_S3_CONNECTION: S3ConnectionFormValues = {
name: "",
bucket: "",
region: "us-east-1",
endpoint: "",
accessKeyId: "",
secretAccessKey: "",
};
export function s3ConnectionRequestConfig(
values: S3ConnectionFormValues,
): Record<string, unknown> {
return {
bucket: values.bucket.trim(),
region: values.region.trim(),
endpoint: values.endpoint.trim(),
accessKeyId: values.accessKeyId.trim(),
secretAccessKey: values.secretAccessKey,
};
}
export function s3ConnectionFormValid(values: S3ConnectionFormValues): boolean {
return (
values.name.trim() !== "" &&
values.bucket.trim() !== "" &&
values.accessKeyId.trim() !== "" &&
values.secretAccessKey.trim() !== ""
);
}
interface S3ConnectionFormProps {
values: S3ConnectionFormValues;
onChange: (values: S3ConnectionFormValues) => void;
}
export function S3ConnectionForm({ values, onChange }: S3ConnectionFormProps) {
const { t } = useTranslation();
const set = (key: keyof S3ConnectionFormValues, value: string) =>
onChange({ ...values, [key]: value });
return (
<div className="portal-sources__connection-form">
<FormField label={t("portal.connections.s3.fields.name")} required>
<Input
value={values.name}
placeholder={t("portal.connections.s3.fields.namePlaceholder")}
onChange={(e) => set("name", e.target.value)}
/>
</FormField>
<FormField
label={t("portal.sources.types.s3.fields.bucket.label")}
required
>
<Input
value={values.bucket}
placeholder="my-company-inbox"
onChange={(e) => set("bucket", e.target.value)}
/>
</FormField>
<FormField label={t("portal.sources.types.s3.fields.region.label")}>
<Input
value={values.region}
placeholder="us-east-1"
onChange={(e) => set("region", e.target.value)}
/>
</FormField>
<FormField
label={t("portal.sources.types.s3.fields.accessKeyId.label")}
required
>
<Input
value={values.accessKeyId}
onChange={(e) => set("accessKeyId", e.target.value)}
/>
</FormField>
<FormField
label={t("portal.sources.types.s3.fields.secretAccessKey.label")}
required
>
<Input
type="password"
value={values.secretAccessKey}
onChange={(e) => set("secretAccessKey", e.target.value)}
/>
</FormField>
<FormField
label={t("portal.sources.types.s3.fields.endpoint.label")}
helperText={t("portal.sources.types.s3.fields.endpoint.helperText")}
>
<Input
value={values.endpoint}
placeholder="https://s3.example.com"
onChange={(e) => set("endpoint", e.target.value)}
/>
</FormField>
</div>
);
}
@@ -0,0 +1,131 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
fireEvent,
render as baseRender,
screen,
waitFor,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { S3ConnectionModal } from "@portal/components/sources/S3ConnectionModal";
import type { IntegrationConfig } from "@portal/api/integrations";
const render = (ui: Parameters<typeof baseRender>[0]) =>
baseRender(ui, { wrapper: MantineProvider });
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => key,
i18n: { changeLanguage: vi.fn() },
}),
}));
const createIntegration = vi.fn();
const updateIntegration = vi.fn();
vi.mock("@portal/api/integrations", () => ({
createIntegration: (...a: unknown[]) => createIntegration(...a),
updateIntegration: (...a: unknown[]) => updateIntegration(...a),
}));
function setField(labelPattern: RegExp, value: string) {
fireEvent.change(screen.getByLabelText(labelPattern), { target: { value } });
}
describe("S3ConnectionModal", () => {
beforeEach(() => {
createIntegration.mockReset();
updateIntegration.mockReset();
});
it("creates a team-scoped connection from the entered fields", async () => {
createIntegration.mockResolvedValue({ id: 5, name: "Claims bucket" });
const onSaved = vi.fn();
const onClose = vi.fn();
render(<S3ConnectionModal open onClose={onClose} onSaved={onSaved} />);
setField(/portal\.connections\.s3\.fields\.name/, "Claims bucket");
setField(/portal\.sources\.types\.s3\.fields\.bucket\.label/, "inbox");
setField(/portal\.sources\.types\.s3\.fields\.accessKeyId\.label/, "AKIA");
setField(
/portal\.sources\.types\.s3\.fields\.secretAccessKey\.label/,
"shh",
);
fireEvent.click(screen.getByText("portal.connections.picker.save"));
await waitFor(() => expect(createIntegration).toHaveBeenCalledTimes(1));
expect(createIntegration).toHaveBeenCalledWith({
integrationType: "S3",
name: "Claims bucket",
scope: "TEAM",
config: {
bucket: "inbox",
region: "us-east-1",
endpoint: "",
accessKeyId: "AKIA",
secretAccessKey: "shh",
},
});
await waitFor(() => expect(onSaved).toHaveBeenCalledTimes(1));
expect(onClose).toHaveBeenCalled();
});
it("round-trips a masked secret unchanged on edit (keeps the stored value)", async () => {
updateIntegration.mockResolvedValue({ id: 5, name: "Claims bucket" });
// The API returns secrets masked; the modal must resend the sentinel verbatim
// so the backend keeps the stored secret rather than overwriting it.
const connection = {
id: 5,
integrationType: "S3",
name: "Claims bucket",
config: {
bucket: "inbox",
region: "us-east-1",
accessKeyId: "AKIA",
secretAccessKey: "********",
},
canManage: true,
} as unknown as IntegrationConfig;
render(
<S3ConnectionModal
open
connection={connection}
onClose={vi.fn()}
onSaved={vi.fn()}
/>,
);
const secret = screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.secretAccessKey\.label/,
) as HTMLInputElement;
expect(secret.value).toBe("********");
// Change only the name; leave the masked secret untouched.
setField(/portal\.connections\.s3\.fields\.name/, "Renamed bucket");
fireEvent.click(screen.getByText("portal.connections.picker.save"));
await waitFor(() => expect(updateIntegration).toHaveBeenCalledTimes(1));
expect(updateIntegration).toHaveBeenCalledWith(
5,
expect.objectContaining({
name: "Renamed bucket",
config: expect.objectContaining({ secretAccessKey: "********" }),
}),
);
});
it("keeps save disabled until the required fields are present", () => {
render(<S3ConnectionModal open onClose={vi.fn()} onSaved={vi.fn()} />);
const save = () =>
screen.getByText("portal.connections.picker.save").closest("button");
expect(save()).toBeDisabled();
setField(/portal\.connections\.s3\.fields\.name/, "Only a name");
expect(save()).toBeDisabled();
setField(/portal\.sources\.types\.s3\.fields\.bucket\.label/, "inbox");
setField(/portal\.sources\.types\.s3\.fields\.accessKeyId\.label/, "AKIA");
setField(
/portal\.sources\.types\.s3\.fields\.secretAccessKey\.label/,
"shh",
);
expect(save()).not.toBeDisabled();
});
});
@@ -0,0 +1,127 @@
import { useEffect, useState } from "react";
import { useTranslation } from "react-i18next";
import { Banner, Button, Modal } from "@app/ui";
import { errorMessage } from "@portal/api/http";
import {
createIntegration,
updateIntegration,
type IntegrationConfig,
} from "@portal/api/integrations";
import {
EMPTY_S3_CONNECTION,
S3ConnectionForm,
s3ConnectionFormValid,
s3ConnectionRequestConfig,
type S3ConnectionFormValues,
} from "@portal/components/sources/S3ConnectionForm";
/**
* The one place S3 connections are created and edited. Launched from the
* Connections tab, the source builder's connection picker, and the pipeline
* builder output - so connection setup is always a modal, never inline splat.
* Saving validates backend-side (schema, SSRF, credentials); on edit the secret
* arrives masked and round-trips unchanged to keep the stored value.
*/
interface S3ConnectionModalProps {
open: boolean;
/** When set, edit this connection; otherwise create a new one. */
connection?: IntegrationConfig | null;
onClose: () => void;
/** The saved connection, so callers can select or refresh it. */
onSaved: (connection: IntegrationConfig) => void;
}
export function S3ConnectionModal({
open,
connection,
onClose,
onSaved,
}: S3ConnectionModalProps) {
const { t } = useTranslation();
const [form, setForm] = useState<S3ConnectionFormValues>(EMPTY_S3_CONNECTION);
const [saving, setSaving] = useState(false);
const [error, setError] = useState<string | null>(null);
const isEdit = Boolean(connection);
// Seed the form each time the modal opens (or its target changes).
useEffect(() => {
if (!open) return;
if (connection) {
const config = connection.config ?? {};
setForm({
name: connection.name,
bucket: String(config.bucket ?? ""),
region: String(config.region ?? "us-east-1"),
endpoint: String(config.endpoint ?? ""),
accessKeyId: String(config.accessKeyId ?? ""),
secretAccessKey: String(config.secretAccessKey ?? ""),
});
} else {
setForm(EMPTY_S3_CONNECTION);
}
setError(null);
}, [open, connection]);
async function save() {
if (saving || !s3ConnectionFormValid(form)) return;
setSaving(true);
setError(null);
try {
const saved = connection
? await updateIntegration(connection.id, {
name: form.name.trim(),
config: s3ConnectionRequestConfig(form),
})
: // TEAM scope suits the team-based portal (the backend defaults the team to the
// caller's own). A teamless single-operator self-hosted deployment would need a
// USER/SERVER scope choice here - follow-up if the portal ships there.
await createIntegration({
integrationType: "S3",
name: form.name.trim(),
scope: "TEAM",
config: s3ConnectionRequestConfig(form),
});
onSaved(saved);
onClose();
} catch (e) {
setError(errorMessage(e));
} finally {
setSaving(false);
}
}
return (
<Modal
open={open}
onClose={onClose}
title={t(
isEdit
? "portal.connections.editTitle"
: "portal.connections.createTitle",
)}
footer={
<div className="portal-sources__connection-create-actions">
<Button
variant="tertiary"
size="sm"
disabled={saving}
onClick={onClose}
>
{t("portal.connections.picker.cancel")}
</Button>
<Button
size="sm"
loading={saving}
disabled={!s3ConnectionFormValid(form)}
onClick={() => void save()}
>
{t("portal.connections.picker.save")}
</Button>
</div>
}
>
<S3ConnectionForm values={form} onChange={setForm} />
{error && <Banner tone="danger" description={error} />}
</Modal>
);
}
@@ -0,0 +1,72 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
fireEvent,
render as baseRender,
screen,
waitFor,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { S3ConnectionPicker } from "@portal/components/sources/S3ConnectionPicker";
const render = (ui: Parameters<typeof baseRender>[0]) =>
baseRender(ui, { wrapper: MantineProvider });
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => key,
i18n: { changeLanguage: vi.fn() },
}),
}));
const fetchS3Connections = vi.fn();
const createIntegration = vi.fn();
vi.mock("@portal/api/integrations", () => ({
fetchS3Connections: () => fetchS3Connections(),
createIntegration: (...a: unknown[]) => createIntegration(...a),
updateIntegration: vi.fn(),
}));
describe("S3ConnectionPicker", () => {
beforeEach(() => {
fetchS3Connections.mockReset();
fetchS3Connections.mockResolvedValue([]);
createIntegration.mockReset();
});
it("creates a connection inline and selects it", async () => {
createIntegration.mockResolvedValue({ id: 7, name: "New bucket" });
const onChange = vi.fn();
render(<S3ConnectionPicker value="" onChange={onChange} />);
fireEvent.click(
await screen.findByText("portal.connections.picker.createNew"),
);
fireEvent.change(
screen.getByLabelText(/portal\.connections\.s3\.fields\.name/),
{ target: { value: "New bucket" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.bucket\.label/,
),
{ target: { value: "inbox" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.accessKeyId\.label/,
),
{ target: { value: "AKIA" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.secretAccessKey\.label/,
),
{ target: { value: "shh" } },
);
fireEvent.click(screen.getByText("portal.connections.picker.save"));
await waitFor(() => expect(createIntegration).toHaveBeenCalledTimes(1));
// The newly created connection's id is selected in the parent.
await waitFor(() => expect(onChange).toHaveBeenCalledWith("7"));
});
});
@@ -0,0 +1,71 @@
import { useEffect, useState } from "react";
import { useTranslation } from "react-i18next";
import { Banner, Button, Select } from "@app/ui";
import { errorMessage } from "@portal/api/http";
import {
fetchS3Connections,
type IntegrationConfig,
} from "@portal/api/integrations";
import { S3ConnectionModal } from "@portal/components/sources/S3ConnectionModal";
/**
* Selects a stored S3 connection by id. Creating a new one opens the shared
* connection modal (saved immediately and validated backend-side), so the
* parent only ever sees a real connection id.
*/
interface S3ConnectionPickerProps {
value: string;
onChange: (connectionId: string) => void;
}
export function S3ConnectionPicker({
value,
onChange,
}: S3ConnectionPickerProps) {
const { t } = useTranslation();
const [connections, setConnections] = useState<IntegrationConfig[] | null>(
null,
);
const [modalOpen, setModalOpen] = useState(false);
const [error, setError] = useState<string | null>(null);
useEffect(() => {
let mounted = true;
fetchS3Connections()
.then((list) => {
if (mounted) setConnections(list);
})
.catch((e) => {
if (mounted) setError(errorMessage(e));
});
return () => {
mounted = false;
};
}, []);
return (
<div className="portal-sources__connection-picker">
<Select
value={value || null}
placeholder={t("portal.connections.picker.placeholder")}
options={(connections ?? []).map((connection) => ({
value: String(connection.id),
label: connection.name,
}))}
onChange={(selected) => onChange(selected ?? "")}
/>
<Button variant="tertiary" size="sm" onClick={() => setModalOpen(true)}>
{t("portal.connections.picker.createNew")}
</Button>
{error && <Banner tone="danger" description={error} />}
<S3ConnectionModal
open={modalOpen}
onClose={() => setModalOpen(false)}
onSaved={(created) => {
setConnections((list) => [...(list ?? []), created]);
onChange(String(created.id));
}}
/>
</div>
);
}
@@ -1,63 +0,0 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import type { SourceView } from "@portal/api/sources";
import { SourceDetailCard } from "@portal/components/sources/SourceDetailCard";
import { sampleDailySeries } from "@portal/mocks/sampleDailySeries";
const SAMPLE_SERIES = sampleDailySeries(330);
const IN_USE: SourceView = {
id: "src-claims",
name: "Claims intake",
type: "folder",
status: "active",
referenceCount: 2,
referencingPolicies: [
{ id: "a", name: "Security Policy" },
{ id: "b", name: "Redaction Policy" },
],
config: [
{ label: "Directory", value: "/data/claims-intake" },
{ label: "Mode", value: "consume" },
],
docsTotal: 45230,
docs24h: 312,
docs30d: 9870,
};
const ORPHANED: SourceView = {
id: "src-archive",
name: "Archive reprocess",
type: "folder",
status: "unused",
referenceCount: 0,
referencingPolicies: [],
config: [{ label: "Directory", value: "/data/archive" }],
docsTotal: 45230,
docs24h: 312,
docs30d: 9870,
};
const meta: Meta<typeof SourceDetailCard> = {
title: "Portal/Sources/SourceDetailCard",
component: SourceDetailCard,
parameters: { layout: "padded" },
args: {
docSeries: SAMPLE_SERIES,
onClose: () => {},
onEdit: () => {},
onTogglePause: () => {},
onDelete: () => {},
},
decorators: [
(S) => (
<div style={{ maxWidth: "56rem" }}>
<S />
</div>
),
],
};
export default meta;
type Story = StoryObj<typeof SourceDetailCard>;
export const InUse: Story = { args: { source: IN_USE } };
export const Orphaned: Story = { args: { source: ORPHANED } };
@@ -1,100 +0,0 @@
import { useTranslation } from "react-i18next";
import { ActionIcon, Button } from "@app/ui";
import type { SourceView } from "@portal/api/sources";
import { SourceDetailPanel } from "@portal/components/sources/SourceDetailPanel";
import {
EDITOR_SOURCE_TYPE,
sourceTypeMeta,
} from "@portal/components/sources/sourceTypes";
import "@portal/views/Sources.css";
import CloseIcon from "@mui/icons-material/Close";
interface SourceDetailCardProps {
source: SourceView;
docSeries: number[];
onClose: () => void;
onEdit: (source: SourceView) => void;
onTogglePause: (source: SourceView) => void;
onDelete: (source: SourceView) => void;
/** Disables the actions while a mutation is in flight. */
busy?: boolean;
}
/** Expanded detail for the selected source row, with edit/pause/delete actions. */
export function SourceDetailCard({
source,
docSeries,
onClose,
onEdit,
onTogglePause,
onDelete,
busy = false,
}: SourceDetailCardProps) {
const { t } = useTranslation();
const meta = sourceTypeMeta(source.type);
const paused = source.status === "disabled";
// The editor is a built-in source: it has no instance name and can't be edited/paused/deleted.
const isEditor = source.type === EDITOR_SOURCE_TYPE;
return (
<section className="portal-sources__expanded">
<header className="portal-sources__expanded-head">
<span
className={`portal-sources__type-dot portal-sources__type-dot--${meta.accent}`}
aria-hidden
>
{meta.icon}
</span>
<div>
<h2 className="portal-sources__expanded-title">
{isEditor ? t(meta.labelKey) : source.name}
</h2>
<span className="portal-sources__expanded-sub">
{t("portal.sources.detail.subtitle", {
type: t(meta.labelKey),
status: t(`portal.sources.status.${source.status}`),
})}
</span>
</div>
<ActionIcon
variant="tertiary"
className="portal-sources__expanded-close"
onClick={onClose}
aria-label={t("portal.sources.detail.closeAriaLabel")}
>
<CloseIcon />
</ActionIcon>
</header>
<SourceDetailPanel source={source} docSeries={docSeries} />
{!isEditor && (
<div className="portal-sources__detail-actions">
<Button
variant="secondary"
disabled={busy}
onClick={() => onEdit(source)}
>
{t("portal.sources.detail.edit")}
</Button>
<Button
variant="secondary"
disabled={busy}
onClick={() => onTogglePause(source)}
>
{paused
? t("portal.sources.detail.resume")
: t("portal.sources.detail.pause")}
</Button>
<Button
accent="danger"
variant="secondary"
disabled={busy}
onClick={() => onDelete(source)}
>
{t("portal.sources.detail.delete")}
</Button>
</div>
)}
</section>
);
}
@@ -1,59 +0,0 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import type { SourceView } from "@portal/api/sources";
import { SourceDetailPanel } from "@portal/components/sources/SourceDetailPanel";
import { sampleDailySeries } from "@portal/mocks/sampleDailySeries";
const SAMPLE_SERIES = sampleDailySeries(330);
const IN_USE: SourceView = {
id: "src-claims",
name: "Claims intake",
type: "folder",
status: "active",
referenceCount: 2,
referencingPolicies: [
{ id: "a", name: "Security Policy" },
{ id: "b", name: "Redaction Policy" },
],
config: [
{ label: "Directory", value: "/data/claims-intake" },
{ label: "Mode", value: "consume" },
],
docsTotal: 45230,
docs24h: 312,
docs30d: 9870,
};
const ORPHANED: SourceView = {
id: "src-archive",
name: "Archive reprocess",
type: "folder",
status: "unused",
referenceCount: 0,
referencingPolicies: [],
config: [{ label: "Directory", value: "/data/archive" }],
docsTotal: 1180,
docs24h: 0,
docs30d: 0,
};
const meta: Meta<typeof SourceDetailPanel> = {
title: "Portal/Sources/SourceDetailPanel",
component: SourceDetailPanel,
parameters: { layout: "padded" },
decorators: [
(S) => (
<div style={{ maxWidth: "48rem" }}>
<S />
</div>
),
],
};
export default meta;
type Story = StoryObj<typeof SourceDetailPanel>;
export const InUse: Story = {
args: { source: IN_USE, docSeries: SAMPLE_SERIES },
};
/** A source no policy references is called out as safe to delete. */
export const Orphaned: Story = { args: { source: ORPHANED, docSeries: [] } };
@@ -1,91 +0,0 @@
import { useTranslation } from "react-i18next";
import { Chip, StatTile } from "@app/ui";
import type { SourceView } from "@portal/api/sources";
import { Sparkline } from "@portal/components/sources/Sparkline";
import { EDITOR_SOURCE_TYPE } from "@portal/components/sources/sourceTypes";
import "@portal/views/Sources.css";
interface SourceDetailPanelProps {
source: SourceView;
/** The 30-day daily series for the sparkline, fetched per source when expanded. */
docSeries: number[];
}
/**
* Expanded detail for a source row: its config (key/value), the documents it has
* fed into runs, and which policies reference it (a 0-reference source is called
* out as safe to delete).
*/
export function SourceDetailPanel({
source,
docSeries,
}: SourceDetailPanelProps) {
const { t } = useTranslation();
const isEditor = source.type === EDITOR_SOURCE_TYPE;
return (
<div className="portal-sources__detail">
{isEditor && (
<p className="portal-sources__muted">
{t("portal.sources.types.editor.description")}
</p>
)}
{source.config.length > 0 && (
<div className="portal-sources__stat-grid">
{source.config.map((row) => (
<StatTile key={row.label} label={row.label} value={row.value} />
))}
</div>
)}
<div className="portal-sources__detail-section">
<span className="portal-sources__detail-heading">
{t("portal.sources.detail.usedBy")}
</span>
{source.referencingPolicies.length === 0 ? (
<p className="portal-sources__muted">
{t(
isEditor
? "portal.sources.types.editor.noPolicies"
: "portal.sources.detail.notReferenced",
)}
</p>
) : (
<div className="portal-sources__chips">
{source.referencingPolicies.map((policy) => (
<Chip key={policy.id} size="sm">
{policy.name}
</Chip>
))}
</div>
)}
</div>
<div className="portal-sources__detail-section">
<span className="portal-sources__detail-heading">
{t("portal.sources.detail.documents")}
</span>
<div className="portal-sources__stat-grid">
<StatTile
label={t("portal.sources.detail.docsTotal")}
value={source.docsTotal.toLocaleString()}
/>
<StatTile
label={t("portal.sources.detail.docs24h")}
value={source.docs24h.toLocaleString()}
/>
<StatTile
label={t("portal.sources.detail.docs30d")}
value={source.docs30d.toLocaleString()}
/>
</div>
{docSeries.length > 0 && (
<Sparkline
data={docSeries}
ariaLabel={t("portal.sources.detail.docsTrend")}
/>
)}
</div>
</div>
);
}
@@ -51,14 +51,9 @@ const meta: Meta<typeof SourcesTable> = {
title: "Portal/Sources/SourcesTable",
component: SourcesTable,
parameters: { layout: "padded" },
args: { sources: SOURCES, expandedId: null, onRowClick: () => {} },
args: { sources: SOURCES, onRowClick: () => {} },
};
export default meta;
type Story = StoryObj<typeof SourcesTable>;
export const Default: Story = {};
/** A row with an open detail panel rotates its caret. */
export const RowExpanded: Story = {
args: { expandedId: SOURCES[0].id },
};
@@ -1,5 +1,6 @@
import { useMemo } from "react";
import { useTranslation } from "react-i18next";
import ChevronRightRoundedIcon from "@mui/icons-material/ChevronRightRounded";
import {
Chip,
StatusBadge,
@@ -22,16 +23,11 @@ const STATUS_TONE: Record<SourceStatus, StatusTone> = {
interface SourcesTableProps {
sources: SourceView[];
/** Id of the row whose detail panel is open, drives the caret state. */
expandedId: string | null;
/** Opens a source's own page. Not called for the virtual editor row. */
onRowClick: (source: SourceView) => void;
}
export function SourcesTable({
sources,
expandedId,
onRowClick,
}: SourcesTableProps) {
export function SourcesTable({ sources, onRowClick }: SourcesTableProps) {
const { t } = useTranslation();
const columns = useMemo<TableColumn<SourceView>[]>(
() => [
@@ -76,6 +72,18 @@ export function SourcesTable({
</StatusBadge>
),
},
{
key: "docs",
header: t("portal.sources.table.documents"),
align: "right",
render: (s) => (
<span
className={s.docsTotal === 0 ? "portal-sources__muted" : undefined}
>
{s.docsTotal.toLocaleString()}
</span>
),
},
{
key: "referenceCount",
header: t("portal.sources.table.usedBy"),
@@ -91,23 +99,20 @@ export function SourcesTable({
),
},
{
key: "expand",
key: "open",
header: "",
align: "right",
width: "2.5rem",
render: (s) => (
<span
className={
"portal-sources__caret" + (expandedId === s.id ? " is-open" : "")
}
aria-hidden
>
▸
</span>
),
// The editor source has no page to open, so it shows no chevron.
render: (s) =>
s.type === EDITOR_SOURCE_TYPE ? null : (
<span className="portal-sources__caret" aria-hidden>
<ChevronRightRoundedIcon style={{ fontSize: "1.25rem" }} />
</span>
),
},
],
[expandedId, t],
[t],
);
return (
@@ -117,6 +122,8 @@ export function SourcesTable({
rows={sources}
rowKey={(s) => s.id}
onRowClick={onRowClick}
// The virtual editor row has no page to open, so it's inert - not a fake button.
isRowInteractive={(s) => s.type !== EDITOR_SOURCE_TYPE}
/>
);
}
@@ -1,48 +0,0 @@
import { describe, expect, it } from "vitest";
import { render } from "@testing-library/react";
import { Sparkline } from "@portal/components/sources/Sparkline";
function pointsOf(container: HTMLElement): string[] {
const poly = container.querySelector("polyline");
return (poly?.getAttribute("points") ?? "")
.trim()
.split(/\s+/)
.filter(Boolean);
}
describe("Sparkline", () => {
it("draws one point per value", () => {
const { container } = render(<Sparkline data={[1, 5, 2, 8, 3]} />);
expect(pointsOf(container)).toHaveLength(5);
});
it("renders nothing for an empty series", () => {
const { container } = render(<Sparkline data={[]} />);
expect(container.querySelector("svg")).toBeNull();
});
it("renders a flat finite line for an all-zero series (no divide-by-zero)", () => {
const { container } = render(<Sparkline data={[0, 0, 0, 0]} />);
const ys = pointsOf(container).map((p) => Number(p.split(",")[1]));
expect(ys).toHaveLength(4);
expect(ys.every(Number.isFinite)).toBe(true);
// All equal: a zero series is a single horizontal line, not NaN-laden.
expect(new Set(ys).size).toBe(1);
});
it("puts the peak value at the top of the band", () => {
const { container } = render(<Sparkline data={[0, 10]} height={36} />);
const ys = pointsOf(container).map((p) => Number(p.split(",")[1]));
// y grows downward in SVG, so the larger value (10) sits at the smaller y.
expect(ys[1]).toBeLessThan(ys[0]);
});
it("exposes its aria-label", () => {
const { container } = render(
<Sparkline data={[1, 2]} ariaLabel="Docs trend" />,
);
expect(container.querySelector("svg")?.getAttribute("aria-label")).toBe(
"Docs trend",
);
});
});
@@ -1,53 +0,0 @@
import "@portal/views/Sources.css";
interface SparklineProps {
/** Series values, oldest first. */
data: number[];
/** Drawing height in px; the width fills the container. */
height?: number;
ariaLabel?: string;
}
/**
* A tiny dependency-free trend line: normalises {@code data} to its own peak and draws a single
* polyline. The viewBox is fixed while the rendered width fills the container (the stroke stays
* crisp via non-scaling-stroke), so it adapts to any column without distorting the line weight.
*/
export function Sparkline({ data, height = 36, ariaLabel }: SparklineProps) {
if (data.length === 0) {
return null;
}
const width = 240;
const pad = 3;
const max = Math.max(...data, 1);
const stepX = data.length > 1 ? width / (data.length - 1) : 0;
const points = data
.map((value, i) => {
const x = i * stepX;
const y = pad + (1 - value / max) * (height - 2 * pad);
return `${x.toFixed(1)},${y.toFixed(1)}`;
})
.join(" ");
return (
<svg
className="portal-sparkline"
width="100%"
height={height}
viewBox={`0 0 ${width} ${height}`}
preserveAspectRatio="none"
role="img"
aria-label={ariaLabel}
>
<polyline
points={points}
fill="none"
stroke="currentColor"
strokeWidth="1.5"
strokeLinejoin="round"
strokeLinecap="round"
vectorEffect="non-scaling-stroke"
/>
</svg>
);
}
@@ -54,7 +54,7 @@ export function sourceTypeMeta(type: string): SourceTypeMeta {
export interface SourceFieldDef {
key: string;
labelKey: string;
control: "text" | "password" | "select";
control: "text" | "password" | "select" | "s3Connection";
required?: boolean;
placeholderKey?: string;
helperTextKey?: string;
@@ -148,18 +148,11 @@ export const CREATABLE_SOURCE_TYPES: CreatableSourceType[] = [
descriptionKey: "portal.sources.types.s3.description",
fields: [
{
key: "bucket",
labelKey: "portal.sources.types.s3.fields.bucket.label",
control: "text",
key: "connectionId",
labelKey: "portal.sources.types.s3.fields.connection.label",
control: "s3Connection",
required: true,
placeholderKey: "portal.sources.types.s3.fields.bucket.placeholder",
},
{
key: "region",
labelKey: "portal.sources.types.s3.fields.region.label",
control: "text",
defaultValue: "us-east-1",
placeholderKey: "portal.sources.types.s3.fields.region.placeholder",
helperTextKey: "portal.sources.types.s3.fields.connection.helperText",
},
{
key: "prefix",
@@ -168,25 +161,6 @@ export const CREATABLE_SOURCE_TYPES: CreatableSourceType[] = [
placeholderKey: "portal.sources.types.s3.fields.prefix.placeholder",
helperTextKey: "portal.sources.types.s3.fields.prefix.helperText",
},
{
key: "accessKeyId",
labelKey: "portal.sources.types.s3.fields.accessKeyId.label",
control: "text",
required: true,
},
{
key: "secretAccessKey",
labelKey: "portal.sources.types.s3.fields.secretAccessKey.label",
control: "password",
required: true,
},
{
key: "endpoint",
labelKey: "portal.sources.types.s3.fields.endpoint.label",
control: "text",
placeholderKey: "portal.sources.types.s3.fields.endpoint.placeholder",
helperTextKey: "portal.sources.types.s3.fields.endpoint.helperText",
},
{
key: "mode",
labelKey: "portal.sources.types.s3.fields.mode.label",
@@ -1,4 +1,5 @@
import { useTranslation } from "react-i18next";
import type { TFunction } from "i18next";
import { Avatar, Button } from "@app/ui";
import type { PendingInvitation } from "@portal/api/users";
import "@portal/views/Users.css";
@@ -11,13 +12,15 @@ interface PendingInvitationsProps {
/** Human "Expires in 3 days" from an ISO expiry; empty when absent, unparseable,
* or already past (the adapter filters expired invites, so no "expired" state). */
function expiryLabel(iso: string | undefined, expiresWord: string): string {
function expiryLabel(iso: string | undefined, t: TFunction): string {
if (!iso) return "";
const ts = Date.parse(iso);
if (!Number.isFinite(ts) || ts <= Date.now()) return "";
const days = Math.round((ts - Date.now()) / 86400000);
if (days === 0) return `${expiresWord} today`;
return `${expiresWord} in ${days === 1 ? "1 day" : `${days} days`}`;
if (days === 0) return t("users.invites.expiresToday", "Expires today");
return t("users.invites.expiresInDays", "Expires in {{count}} days", {
count: days,
});
}
/**
@@ -30,7 +33,6 @@ export function PendingInvitations({
onCancel,
}: PendingInvitationsProps) {
const { t } = useTranslation();
const expiresWord = t("users.invites.expires", "Expires");
return (
<section className="portal-users__group">
<header className="portal-users__group-head">
@@ -50,7 +52,7 @@ export function PendingInvitations({
</span>
</header>
{invitations.map((inv) => {
const expires = expiryLabel(inv.expiresAt, expiresWord);
const expires = expiryLabel(inv.expiresAt, t);
return (
<div className="portal-users__row" key={inv.id}>
<div className="portal-users__row-main">
@@ -10,16 +10,20 @@ import { usePlanTier } from "@portal/contexts/usePlanTier";
export type Tier = "free" | "pro" | "enterprise";
export interface TierInfo {
label: string;
/** i18n key for the plan label; resolve with `t()` at the call site. */
labelKey: string;
dotColor: string;
}
export const TIER_INFO: Record<Tier, TierInfo> = {
// Matches SaaS branding (editor/cloud Payg + PaygFree): the always-free
// manual-tools tier is "Editor plan"; the metered tier is "Processor plan".
free: { label: "Editor plan", dotColor: "var(--color-text-4)" },
pro: { label: "Processor plan", dotColor: "var(--color-blue)" },
enterprise: { label: "Enterprise plan", dotColor: "var(--color-purple)" },
free: { labelKey: "portal.tier.free", dotColor: "var(--color-text-4)" },
pro: { labelKey: "portal.tier.pro", dotColor: "var(--color-blue)" },
enterprise: {
labelKey: "portal.tier.enterprise",
dotColor: "var(--color-purple)",
},
};
interface TierContextValue {
+91 -48
View File
@@ -4,13 +4,33 @@
* only builds seed data for the MSW handlers and tests.
*/
import type { PolicyRunView, WirePolicy } from "@app/policies/types";
import { POLICY_CONFIG } from "@portal/api/policies";
import type {
PolicyRunView,
WirePipelineStep,
WirePolicy,
} from "@app/policies/types";
/* ──────────────────────────────────────────────────────────────────────── */
/* Seed data — real backend wire format */
/* ──────────────────────────────────────────────────────────────────────── */
// Literal wire steps (not derived from the catalogue) so this fixtures module stays independent of
// @portal/api/policies and its heavy tool-operation import graph.
const SECURITY_STEPS: WirePipelineStep[] = [
{
operation: "/api/v1/security/auto-redact",
parameters: {
listOfText: "",
useRegex: true,
convertPDFToImage: true,
},
},
{
operation: "/api/v1/security/sanitize-pdf",
parameters: { removeJavaScript: true },
},
];
export function seedPolicies(): WirePolicy[] {
return [
{
@@ -19,7 +39,7 @@ export function seedPolicies(): WirePolicy[] {
owner: "security@acme.com",
enabled: true,
trigger: null,
steps: POLICY_CONFIG.security.defaultOperations,
steps: SECURITY_STEPS,
output: {
type: "inline",
options: {
@@ -37,57 +57,80 @@ export function seedPolicies(): WirePolicy[] {
},
},
},
{
id: "pol_ingestion_default",
name: "Ingestion Policy",
owner: "data-eng@acme.com",
enabled: true,
trigger: null,
steps: POLICY_CONFIG.ingestion.defaultOperations,
output: {
type: "inline",
options: {
runOn: "upload",
mode: "new_version",
name: "",
position: "suffix",
maxRetries: 3,
retryDelayMinutes: 5,
categoryId: "ingestion",
sources: ["src-contracts"],
scopeTypes: [],
reviewerEmail: "data-eng@acme.com",
fieldValues: {},
},
},
},
];
}
const NOW = Date.now();
const M = 60000;
const H = 3600000;
const D = 86400000;
/** Seed `PolicyRunView` records that drive the activity feed + stats. */
/** Seed `PolicyRunView` records that drive the activity feed + stats.
* 40 delivered + 3 failed within the trailing 24h so the home visualiser shows
* a lively flow; a tail of older completed runs keeps the lifetime stats real.
* Runs are split across the two active policies so the Sankey waist divides. */
export function seedPolicyRuns(): PolicyRunView[] {
return [
{
runId: "run_001",
policyId: "pol_security_default",
status: "COMPLETED",
currentStep: 2,
stepCount: 2,
error: null,
outputs: [{ fileId: "f1", fileName: "Q2-vendor-agreement.pdf" }],
createdAt: NOW - 12 * M,
},
{
runId: "run_002",
policyId: "pol_security_default",
status: "FAILED",
currentStep: 1,
stepCount: 2,
error: "Low-confidence match — routed for review",
outputs: [{ fileId: "f2", fileName: "patient-intake-0481.pdf" }],
createdAt: NOW - 1 * H,
},
{
runId: "run_003",
policyId: "pol_security_default",
status: "RUNNING",
currentStep: 1,
stepCount: 2,
error: null,
outputs: [{ fileId: "f3", fileName: "invoice-7782.pdf" }],
createdAt: NOW - 2 * M,
},
// Older completed runs for stats
...Array.from({ length: 4818 }, (_, i) => ({
runId: `run_old_${i}`,
policyId: "pol_security_default",
status: "COMPLETED" as const,
currentStep: 2,
stepCount: 2,
error: null,
outputs: [] as { fileId: string; fileName: string }[],
createdAt: NOW - (34 * D + i * 10 * M),
})),
];
// Split the throughput across the two active policies (security / ingestion)
// so both show a 24h count and the Sankey waist splits into two segments.
const policyFor = (i: number, total: number) =>
i < Math.round(total * 0.6)
? "pol_security_default"
: "pol_ingestion_default";
// 40 successful runs spread across the last ~13h.
const delivered = Array.from({ length: 40 }, (_, i) => ({
runId: `run_ok_${i}`,
policyId: policyFor(i, 40),
status: "COMPLETED" as const,
currentStep: 2,
stepCount: 2,
error: null,
outputs: [{ fileId: `f${i}`, fileName: `document-${i + 1}.pdf` }],
createdAt: NOW - i * 20 * M,
}));
// 3 failures within the last few hours.
const failed = Array.from({ length: 3 }, (_, i) => ({
runId: `run_fail_${i}`,
policyId: policyFor(i, 3),
status: "FAILED" as const,
currentStep: 1,
stepCount: 2,
error: "Low-confidence match — routed for review",
outputs: [{ fileId: `ff${i}`, fileName: `flagged-${i + 1}.pdf` }],
createdAt: NOW - (i + 1) * 90 * M,
}));
// Older completed runs (>24h) for lifetime stats — excluded from 24h counts.
const older = Array.from({ length: 4800 }, (_, i) => ({
runId: `run_old_${i}`,
policyId: "pol_security_default",
status: "COMPLETED" as const,
currentStep: 2,
stepCount: 2,
error: null,
outputs: [] as { fileId: string; fileName: string }[],
createdAt: NOW - (34 * D + i * 10 * M),
}));
return [...delivered, ...failed, ...older];
}
@@ -51,7 +51,7 @@ const ENTERPRISE_DEAL: Deal = {
const ENTERPRISE_LEDGER: LedgerGroup[] = [
{
stage: "trial",
label: "Trial",
label: "portal.procurement.journeySteps.trial.label",
docs: [
{
id: "doc-trial-quickstart",
@@ -71,7 +71,7 @@ const ENTERPRISE_LEDGER: LedgerGroup[] = [
},
{
stage: "quote",
label: "Quote",
label: "portal.procurement.journeySteps.quote.label",
docs: [
{
id: "doc-quote-formal",
@@ -84,7 +84,7 @@ const ENTERPRISE_LEDGER: LedgerGroup[] = [
},
{
stage: "security",
label: "Agreement",
label: "portal.procurement.journeySteps.agreement.label",
docs: [
{
id: "doc-agreement-enterprise",
@@ -97,7 +97,7 @@ const ENTERPRISE_LEDGER: LedgerGroup[] = [
},
{
stage: "procurement",
label: "Payment",
label: "portal.procurement.journeySteps.payment.label",
docs: [
{
id: "doc-pay-online",
@@ -124,7 +124,7 @@ const ENTERPRISE_LEDGER: LedgerGroup[] = [
},
{
stage: "active",
label: "Implementation",
label: "portal.procurement.journeySteps.implementation.label",
docs: [
{
id: "doc-active-playbook",
+10 -3
View File
@@ -93,8 +93,11 @@
width: 1.75rem;
height: 1.75rem;
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 600;
flex-shrink: 0;
}
.portal-home__quick-icon svg {
width: 1.05rem;
height: 1.05rem;
}
.portal-home__quick-text {
@@ -116,9 +119,13 @@
}
.portal-home__quick-arrow {
font-size: 0.875rem;
display: inline-flex;
color: var(--color-text-5);
}
.portal-home__quick-arrow svg {
width: 1rem;
height: 1rem;
}
.portal-home__quick-row:hover .portal-home__quick-arrow {
color: var(--color-blue);
+38 -6
View File
@@ -4,6 +4,7 @@ import { useTier } from "@portal/contexts/TierContext";
import { useView, type ViewId } from "@portal/contexts/ViewContext";
import { HomeHero } from "@portal/components/HomeHero";
import { HomeGreeting } from "@portal/components/HomeGreeting";
import { ProcessorFlow } from "@portal/components/ProcessorFlow";
import { RecentActivity } from "@portal/components/RecentActivity";
import { ProcessingStatusStrip } from "@portal/components/ProcessingStatusStrip";
import { PolicySummary } from "@portal/components/PolicySummary";
@@ -13,31 +14,52 @@ import "@portal/views/Home.css";
/* Quick actions card */
/* ──────────────────────────────────────────────────────────────────────── */
/** A stroke icon used inside the quick-action badge (replaces the old text
glyphs ⌃ ⇢ ⚙, which rendered off-style vs the portal's icon set). */
function QuickIcon({ d }: { d: string }) {
return (
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="1.7"
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden
>
<path d={d} />
</svg>
);
}
/** Rows for the Quick Actions list. Each `view` navigates the portal. */
const QUICK_ACTIONS: Array<{
key: string;
glyph: string;
iconD: string;
bg: string;
fg: string;
view: ViewId;
}> = [
{
key: "buildPipeline",
glyph: "⌃",
iconD:
"M6 4a2 2 0 100 4 2 2 0 000-4zM18 16a2 2 0 100 4 2 2 0 000-4zM6 8v6a4 4 0 004 4h4",
bg: "var(--color-purple-light)",
fg: "var(--color-purple)",
view: "pipelines",
},
{
key: "connectSource",
glyph: "⇢",
iconD:
"M10 13a5 5 0 007 0l3-3a5 5 0 00-7-7l-1 1M14 11a5 5 0 00-7 0l-3 3a5 5 0 007 7l1-1",
bg: "var(--color-green-light)",
fg: "var(--color-green-dark)",
view: "sources",
},
{
key: "issueApiKey",
glyph: "⚙",
iconD:
"M2.6 17.4A2 2 0 002 18.8V21a1 1 0 001 1h3a1 1 0 001-1v-1a1 1 0 011-1h1a1 1 0 001-1v-1a1 1 0 011-1h.2a2 2 0 001.4-.6l.8-.8a6.5 6.5 0 10-4-4z M16.5 7.5 h.01",
bg: "var(--color-amber-light)",
fg: "var(--color-amber-dark)",
view: "infrastructure",
@@ -75,12 +97,21 @@ function QuickActions() {
style={{ background: action.bg, color: action.fg }}
aria-hidden
>
{action.glyph}
<QuickIcon d={action.iconD} />
</span>
}
rightSection={
<span className="portal-home__quick-arrow" aria-hidden>
→
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
strokeLinecap="round"
strokeLinejoin="round"
>
<path d="M9 6l6 6-6 6" />
</svg>
</span>
}
>
@@ -112,6 +143,7 @@ export function Home() {
{/* Per-tier hero. Its footer is the deal-status hero while a procurement
deal is underway (a bolt-on to any tier), otherwise the setup checklist. */}
<HomeHero tier={tier} />
<ProcessorFlow />
{/* One unified layout across tiers: real processed-PDF volume, real audit
activity, quick actions, and the standing-policy summary. */}
@@ -1,4 +1,5 @@
import { useState } from "react";
import { useEffect, useState } from "react";
import { useSearchParams } from "react-router-dom";
import { useTranslation } from "react-i18next";
import { Button, Tabs, type TabItem } from "@app/ui";
import { useView } from "@portal/contexts/ViewContext";
@@ -18,10 +19,33 @@ type InfraTab =
| "storage"
| "audit";
const INFRA_TABS: InfraTab[] = [
"deployments",
"api-keys",
"security",
"models",
"storage",
"audit",
];
export function Infrastructure() {
const { t } = useTranslation();
const [tab, setTab] = useState<InfraTab>("deployments");
const { setActiveView } = useView();
const [searchParams, setSearchParams] = useSearchParams();
// Deep-link (?tab=<key>) from elsewhere (e.g. the home visualiser's outcome
// cards → audit log): open that tab, then drop the param.
useEffect(() => {
const requested = searchParams.get("tab");
if (!requested) return;
if ((INFRA_TABS as string[]).includes(requested)) {
setTab(requested as InfraTab);
}
const next = new URLSearchParams(searchParams);
next.delete("tab");
setSearchParams(next, { replace: true });
}, [searchParams, setSearchParams]);
const tabs: TabItem<InfraTab>[] = [
{ key: "deployments", label: t("portal.infrastructure.tabs.deployments") },
@@ -392,27 +392,3 @@
min-width: 1.5rem;
min-height: 1.5rem;
}
.portal-builder__s3-output {
display: flex;
align-items: center;
gap: 0.625rem;
}
.portal-builder__s3-summary {
font-size: 0.8125rem;
color: var(--color-text-1);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.portal-builder__s3-summary.is-unset {
color: var(--color-text-4);
}
.portal-builder__s3-fields {
display: flex;
flex-direction: column;
gap: 0.75rem;
}
@@ -50,6 +50,13 @@ vi.mock("@portal/api/policies", () => ({
clearProcessedHistory: (id: string) => clearProcessedHistory(id),
}));
const fetchS3Connections = vi.fn();
const createIntegration = vi.fn();
vi.mock("@portal/api/integrations", () => ({
fetchS3Connections: () => fetchS3Connections(),
createIntegration: (...args: unknown[]) => createIntegration(...args),
}));
// One editable tool, Compress, so the picker and step settings have something to render.
vi.mock("@app/contexts/ToolRegistryContext", () => {
const compress = {
@@ -149,6 +156,9 @@ describe("PipelineBuilder", () => {
fetchRun.mockResolvedValue({ status: "COMPLETED" });
clearProcessedHistory.mockReset();
clearProcessedHistory.mockResolvedValue(undefined);
fetchS3Connections.mockReset();
fetchS3Connections.mockResolvedValue([]);
createIntegration.mockReset();
});
it("builds a new pipeline: name it, add a tool, and save", async () => {
@@ -177,7 +187,8 @@ describe("PipelineBuilder", () => {
expect(await screen.findByText("pipelines list")).toBeInTheDocument();
});
it("saves an s3 output with its connection options", async () => {
it("saves an s3 output referencing an inline-created connection", async () => {
createIntegration.mockResolvedValue({ id: 12, name: "Claims bucket" });
renderBuilder("/processor/pipelines/new");
fireEvent.change(await screen.findByRole("textbox"), {
@@ -185,32 +196,55 @@ describe("PipelineBuilder", () => {
});
fireEvent.click(screen.getByLabelText("portal.pipelines.output.s3"));
// With s3 selected but no bucket, saving is blocked and the summary reads
// unconfigured; the connection fields live behind the Configure modal.
// With s3 selected but no connection chosen, saving is blocked. The
// connection picker + prefix are inline (no modal), like the folder output.
expect(
screen.getByText("portal.pipelines.composer.create").closest("button"),
).toBeDisabled();
expect(
screen.getByText("portal.pipelines.composer.s3NotConfigured"),
).toBeInTheDocument();
fireEvent.click(screen.getByText("portal.pipelines.composer.s3Configure"));
// Textboxes: name, then the modal's bucket, region, prefix, access key id,
// endpoint; the secret renders as a password input outside the textbox role.
const inputs = screen.getAllByRole("textbox") as HTMLInputElement[];
fireEvent.change(inputs[1], { target: { value: "claims-processed" } });
fireEvent.change(inputs[3], { target: { value: "processed/" } });
fireEvent.change(inputs[4], { target: { value: "AKIAEXAMPLE" } });
const secret = document.querySelector(
'input[type="password"]',
) as HTMLInputElement;
fireEvent.change(secret, { target: { value: "shh-secret" } });
fireEvent.click(screen.getByText("portal.pipelines.composer.s3Done"));
// No connections exist: create one inline from the picker. Target fields by
// label, not position - the picker's Mantine Select also carries an input
// role and would shift index-based queries.
fireEvent.click(
await screen.findByText("portal.connections.picker.createNew"),
);
fireEvent.change(
screen.getByLabelText(/portal\.connections\.s3\.fields\.name/),
{ target: { value: "Claims bucket" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.bucket\.label/,
),
{ target: { value: "claims-processed" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.accessKeyId\.label/,
),
{ target: { value: "AKIAEXAMPLE" } },
);
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.secretAccessKey\.label/,
),
{ target: { value: "shh-secret" } },
);
fireEvent.click(screen.getByText("portal.connections.picker.save"));
await waitFor(() => expect(createIntegration).toHaveBeenCalledTimes(1));
// The connection modal closes once saved and the connection is selected.
await waitFor(() =>
expect(
screen.queryByText("portal.connections.picker.save"),
).not.toBeInTheDocument(),
);
// The summary now shows the configured destination.
expect(
screen.getByText("s3://claims-processed/processed/"),
).toBeInTheDocument();
fireEvent.change(
screen.getByLabelText(
/portal\.sources\.types\.s3\.fields\.prefix\.label/,
),
{ target: { value: "processed/" } },
);
fireEvent.click(screen.getByText("portal.pipelines.composer.create"));
await waitFor(() => expect(savePipeline).toHaveBeenCalledTimes(1));
@@ -219,12 +253,8 @@ describe("PipelineBuilder", () => {
output: {
type: "s3",
options: {
bucket: "claims-processed",
region: "us-east-1",
connectionId: "12",
prefix: "processed/",
endpoint: "",
accessKeyId: "AKIAEXAMPLE",
secretAccessKey: "shh-secret",
},
},
}),

Some files were not shown because too many files have changed in this diff Show More