feat: release type and tag filters for mirrored releases (#471)

Adds a Pre-releases switch (stable only), include and exclude tag patterns (globs or /regex/) with a preview of what a save keeps or removes, per organization and per repository overrides, and RELEASE_TYPE, RELEASE_INCLUDE_TAGS and RELEASE_EXCLUDE_TAGS. Releases in the destination that stop matching are removed on the next sync, and a warning is raised when that empties a destination.
This commit is contained in:
Florian Zevedei
2026-10-23 07:58:44 +05:30
committed by GitHub
parent 5b1ef47e20
commit 56d0ef9020
40 changed files with 3677 additions and 223 deletions
+3
View File
@@ -141,6 +141,9 @@ DOCKER_TAG=latest
# Release and Metadata
# MIRROR_RELEASES=false # Mirror GitHub releases
# RELEASE_LIMIT=10 # Maximum number of releases to mirror per repository
# RELEASE_TYPE=all # Which releases to mirror: all, or stable (skip pre-releases)
# RELEASE_INCLUDE_TAGS=v* # Only mirror releases whose tag matches these comma separated patterns
# RELEASE_EXCLUDE_TAGS=*-rc.* # Skip releases whose tag matches; exclude wins over include
# MIRROR_WIKI=false # Mirror wiki content
# Issue Tracking (requires MIRROR_METADATA=true)
+1
View File
@@ -54,3 +54,4 @@ tests/e2e/git-repos/
/playwright/.cache/
/playwright/.auth/
.playwright-mcp/
/.vs
+15
View File
@@ -32,6 +32,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Raised the dependency floors for the advisories published on 2026-09-08: Astro 7.2.8 (remote code execution through AVIF image optimization, and an authorization bypass when stripping the configured base), @xmldom/xmldom 0.8.15 (eight parser and serializer issues), sharp 0.35.4 (libheif), svgo 4.1.0 (removeScripts sanitization) and js-yaml 4.3.2 (merge-key CPU use). Applied to both the application and the documentation site.
### Added
- Choose which releases count as candidates: every one, or stable releases only
- A **Pre-releases** switch sits under the two release numbers in the mirror settings, and the same choice is in the per organization and per repository **Mirror options** dialog, where **Inherit** means the global value
- The release controls in the global mirror settings are listed under each other instead of beside the switch, and in the **Mirror options** dialog the release limit, asset limit and tag rules sit directly under the **Releases** row instead of on a separate tab
- **Stable only** skips everything the source marks as a pre-release, and the default **All releases** keeps the previous behaviour
- The filter runs before the release limit and looks further back first, so "the newest 10" means the newest 10 matching releases rather than whatever survives of the newest 10
- The retention pass also removes destination releases the filter excludes; a release the destination reports no pre-release flag for is kept, so an unknown flag never deletes the wrong thing
- `RELEASE_TYPE=all|stable` seeds the global default, and no migration is needed because the value lives inside the existing JSON columns
- Filter mirrored releases by tag name with **Include tags** and **Exclude tags**
- **Custom tag rules...** opens a dialog holding the two lists, both in the mirror settings and in the per organization and per repository **Mirror options** dialog, where leaving both empty means the level above
- A pattern is a glob over the whole tag (`v*`, `v1.*`, `*-rc.*`, `?` for one character) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively; exclude wins when both lists name the same tag, and a pattern that cannot compile is ignored instead of failing the run. Commas inside the slashes belong to the expression (`/^v\d{1,3}\.\d+$/`), in the dialog and in the environment variables alike
- This is the filter for projects that ship release candidates without marking them as pre-releases, which the type filter cannot separate
- A tag filter widens the release scan like the type filter does, so the limit counts the releases that match, and the retention pass removes destination releases whose tag no longer matches, including ones the destination reports no pre-release flag for
- When the filters delete every release the destination had, the activity log shows a highlighted **warning** entry "All releases removed" and the **Sync errors** switch sends the same alert, so an over-broad rule cannot empty the destination silently
- The dialog lists the releases the destination already holds and crosses out the ones the draft rules would delete on the next sync, so a pattern can be checked before saving. Opened from a repository it previews that one; from an organization or the global settings it adds a picker over the repositories the rules cover
- `RELEASE_INCLUDE_TAGS` and `RELEASE_EXCLUDE_TAGS` seed the global lists; the values live in the existing JSON columns, so no migration is needed
- Notifications for newly discovered repositories
- "New repositories discovered" on the Notifications page, with three sub-switches under it: new repositories in your own account, in one of your organizations, and starred repositories
- The scheduled sync, the boot auto-start and Sync Organization send them once per group with the repositories that were imported; all stars share a single "Starred" alert because their owners are arbitrary, and a long list is cut to the first 10 names plus an "and N more" line
+3 -1
View File
@@ -240,7 +240,9 @@ Transfer complete repository metadata from GitHub to Gitea/Forgejo:
- **Wiki** - Mirror wiki content
- **Releases** - Transfer GitHub releases with assets
Releases have two limits. The release limit keeps only the newest N releases in Gitea/Forgejo. The release asset limit uploads assets only for the newest N of those, so a repository with years of releases can keep every release note and tag while only the recent binaries take up space. Leave the asset limit empty to upload assets for every mirrored release, or set it to 0 for release notes only. Both limits can be overridden per organization and per repository, and lowering the asset limit never deletes assets that were already uploaded.
Releases have two limits and two filters. The release limit keeps only the newest N releases in Gitea/Forgejo. The release asset limit uploads assets only for the newest N of those, so a repository with years of releases can keep every release note and tag while only the recent binaries take up space. Leave the asset limit empty to upload assets for every mirrored release, or set it to 0 for release notes only. The **Pre-releases** switch skips everything the source marks as a pre-release, and **Custom tag rules** narrows releases by tag name (`v1.*`, `*-rc.*`, or a regular expression in slashes) for projects that ship candidates without marking them.
All of it can be overridden per organization and per repository in the **Mirror options** dialog, and lowering the asset limit never deletes assets that were already uploaded.
Enable in Settings → Mirror Options → Mirror metadata
+3
View File
@@ -213,6 +213,9 @@ Control what content gets mirrored from GitHub to Gitea.
|----------|-------------|---------|---------|
| `MIRROR_RELEASES` | Mirror GitHub releases | `false` | `true`, `false` |
| `RELEASE_LIMIT` | Maximum number of releases to mirror per repository | `10` | Number (1-100) |
| `RELEASE_TYPE` | Which releases are candidates: `all` mirrors everything, `stable` skips what the source marks as a pre-release. With a filter active the release limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync | `all` | `all`, `stable` |
| `RELEASE_INCLUDE_TAGS` | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`, `?` for one character) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. Commas inside the slashes belong to the expression (`/^v\d{1,3}\.\d+$/`) | empty | any list of globs or `/regex/` |
| `RELEASE_EXCLUDE_TAGS` | Comma separated tag patterns in the same syntax; releases whose tag matches are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS` | empty | any list of globs or `/regex/` |
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded | unset (all) | Number (0 or more) |
| `MIRROR_WIKI` | Mirror wiki content | `false` | `true`, `false` |
| `MIRROR_METADATA` | Master toggle for metadata mirroring | `false` | `true`, `false` |
+8
View File
@@ -12,6 +12,7 @@ import {
Building2,
ChevronDown,
GitFork,
TriangleAlert,
} from 'lucide-react';
import { Card } from '../ui/card';
import { cn, formatDate, formatLastSyncTime } from '@/lib/utils';
@@ -83,6 +84,13 @@ export const STATUS_PRESENTATION: Record<
tone: 'text-blue-600 dark:text-blue-400',
wash: 'bg-blue-500/10',
},
warning: {
Icon: TriangleAlert,
label: 'Configuration warning',
short: 'warning',
tone: 'text-amber-600 dark:text-amber-400',
wash: 'bg-amber-500/10',
},
};
const FALLBACK_PRESENTATION = {
+9 -7
View File
@@ -29,7 +29,7 @@ import {
SelectTrigger,
SelectValue,
} from '../ui/select';
import { repoStatusEnum, type RepoStatus } from '@/types/Repository';
import { activityStatusEnum, type ActivityStatus } from '@/types/Repository';
import ActivityList, { STATUS_PRESENTATION } from './ActivityList';
import { ActivityNameCombobox } from './ActivityNameCombobox';
import { useSSE } from '@/hooks/useSEE';
@@ -413,7 +413,7 @@ export function ActivityLog() {
const statusCounts = useMemo(() => countLatestStatuses(activities), [activities]);
// Most actionable first, and only statuses that actually occurred.
const statChips = ['failed', 'syncing', 'mirroring', 'imported', 'synced', 'mirrored']
const statChips = ['failed', 'warning', 'syncing', 'mirroring', 'imported', 'synced', 'mirrored']
.filter((status) => (statusCounts.get(status) ?? 0) > 0)
.slice(0, 4);
@@ -507,7 +507,7 @@ export function ActivityLog() {
onValueChange={(v) =>
setFilter((p) => ({
...p,
status: v === 'all' ? '' : (v as RepoStatus),
status: v === 'all' ? '' : (v as ActivityStatus),
}))
}
>
@@ -515,13 +515,14 @@ export function ActivityLog() {
<SelectValue placeholder="All statuses" />
</SelectTrigger>
<SelectContent>
{['all', ...repoStatusEnum.options].map((s) => (
{['all', ...activityStatusEnum.options].map((s) => (
<SelectItem key={s} value={s}>
<span className="flex items-center gap-2">
{s !== 'all' && (
<span className={`h-2 w-2 rounded-full ${
s === 'synced' ? 'bg-green-500' :
s === 'failed' ? 'bg-red-500' :
s === 'warning' ? 'bg-amber-500' :
s === 'syncing' ? 'bg-blue-500' :
'bg-yellow-500'
}`} />
@@ -735,7 +736,7 @@ export function ActivityLog() {
onClick={() =>
setFilter((p) => ({
...p,
status: p.status === status ? '' : (status as RepoStatus),
status: p.status === status ? '' : (status as ActivityStatus),
}))
}
className={cn(
@@ -766,7 +767,7 @@ export function ActivityLog() {
onValueChange={(v) =>
setFilter((p) => ({
...p,
status: v === 'all' ? '' : (v as RepoStatus),
status: v === 'all' ? '' : (v as ActivityStatus),
}))
}
>
@@ -774,13 +775,14 @@ export function ActivityLog() {
<SelectValue placeholder="All statuses" />
</SelectTrigger>
<SelectContent>
{['all', ...repoStatusEnum.options].map((s) => (
{['all', ...activityStatusEnum.options].map((s) => (
<SelectItem key={s} value={s}>
<span className="flex items-center gap-2">
{s !== 'all' && (
<span className={`h-2 w-2 rounded-full ${
s === 'synced' ? 'bg-green-500' :
s === 'failed' ? 'bg-red-500' :
s === 'warning' ? 'bg-amber-500' :
s === 'syncing' ? 'bg-blue-500' :
'bg-yellow-500'
}`} />
+104 -5
View File
@@ -37,7 +37,8 @@ import {
Plus,
Users,
UserX,
X
X,
FlaskConical,
} from "lucide-react";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy, DestinationProvider } from "@/types/config";
import { DESTINATION_PROVIDER_LABELS, isPushDestinationKind } from "@/lib/destination-kinds";
@@ -58,6 +59,7 @@ import {
} from "@/components/ui/command";
import { cn } from "@/lib/utils";
import { githubApi } from "@/lib/api";
import { withBase } from "@/lib/base-path";
import {
sourceKindSupportsReleases,
SOURCE_PROVIDER_LABELS,
@@ -72,6 +74,7 @@ import {
CardDivider,
CardSection,
} from "./settings-ui";
import { ReleaseTagRulesDialog } from "./ReleaseTagRulesDialog";
interface GitHubMirrorSettingsProps {
githubConfig: GitHubConfig;
@@ -133,6 +136,40 @@ export function GitHubMirrorSettings({
const [loadedStarLists, setLoadedStarLists] = React.useState(false);
const [attemptedStarListLoad, setAttemptedStarListLoad] = React.useState(false);
// The tag patterns are edited in a dialog rather than inline: the wildcard
// and regex syntax, and the fact that a now-unmatching release is deleted
// from Gitea, both need room to explain themselves.
const [tagRulesOpen, setTagRulesOpen] = React.useState(false);
const tagPatternCount =
(mirrorOptions.releaseIncludeTags?.length ?? 0) +
(mirrorOptions.releaseExcludeTags?.length ?? 0);
// Global rules hit every repository, so the tag dialog also offers the
// preview picker here, listing the whole account. The list is fetched once,
// lazily, the first time the dialog opens.
const [globalPreviewRepos, setGlobalPreviewRepos] = React.useState<
{ id: string; label: string }[] | null
>(null);
React.useEffect(() => {
if (!tagRulesOpen || globalPreviewRepos !== null) return;
let cancelled = false;
fetch(withBase("/api/repositories/preview-repositories"))
.then((res) => (res.ok ? res.json() : null))
.then((payload) => {
if (cancelled) return;
const data = payload?.data ?? payload;
setGlobalPreviewRepos(
Array.isArray(data?.repositories) ? data.repositories : []
);
})
.catch(() => {
if (!cancelled) setGlobalPreviewRepos([]);
});
return () => {
cancelled = true;
};
}, [tagRulesOpen, globalPreviewRepos]);
const normalizeStarListNames = React.useCallback((lists: string[] | undefined): string[] => {
if (!Array.isArray(lists)) return [];
@@ -164,7 +201,10 @@ export function GitHubMirrorSettings({
onGitHubConfigChange({ ...githubConfig, [field]: value });
};
const handleMirrorChange = (field: keyof MirrorOptions, value: boolean | number | null) => {
const handleMirrorChange = (
field: keyof MirrorOptions,
value: boolean | number | string | string[] | null,
) => {
onMirrorOptionsChange({ ...mirrorOptions, [field]: value });
};
@@ -912,6 +952,7 @@ export function GitHubMirrorSettings({
<SwitchRow
icon={Tag}
align="start"
label="Releases & tags"
description={
isPushTarget
@@ -927,8 +968,8 @@ export function GitHubMirrorSettings({
onCheckedChange={(checked) => handleMirrorChange('mirrorReleases', checked)}
extra={
mirrorOptions.mirrorReleases ? (
<div className="flex flex-wrap items-center justify-end gap-x-3 gap-y-1.5 text-xs text-muted-foreground">
<div className="flex items-center gap-1.5">
<div className="space-y-1.5 text-xs text-muted-foreground">
<div className="flex items-center gap-2">
<label htmlFor="release-limit">Latest</label>
<input
id="release-limit"
@@ -945,7 +986,7 @@ export function GitHubMirrorSettings({
</div>
{/* Empty means every mirrored release gets its assets (the
behaviour before #311); 0 means notes only. */}
<div className="flex items-center gap-1.5">
<div className="flex items-center gap-2">
<label htmlFor="release-asset-limit">Assets for latest</label>
<input
id="release-asset-limit"
@@ -974,6 +1015,46 @@ export function GitHubMirrorSettings({
}
/>
{/* Which kind of release counts as a candidate. Its own row because it
only means anything once releases are mirrored; the tag patterns
live in a dialog because their syntax needs room. */}
{mirrorOptions.mirrorReleases && (
<SwitchRow
icon={FlaskConical}
className="pl-6"
label="Pre-releases"
badge="NEW"
description={
isPushTarget
? pushTargetReason
: !isReleaseCapableSource
? "Release filtering needs a GitHub or Gitea/Forgejo source"
: "Turn off to skip releases the source marks as pre-release. Ones already in Gitea are removed on the next sync"
}
disabled={!isReleaseCapableSource}
checked={mirrorOptions.releaseType !== "stable"}
onCheckedChange={(checked) => handleMirrorChange('releaseType', checked ? "all" : "stable")}
subContent={
<div className="flex items-center gap-2 pt-0.5">
<Button
type="button"
variant="link"
size="sm"
className="h-auto p-0 text-[13px] font-normal"
onClick={() => setTagRulesOpen(true)}
>
Custom tag rules...
</Button>
{tagPatternCount > 0 && (
<span className="rounded-full bg-muted px-2 py-0.5 text-[10px] font-semibold text-muted-foreground">
{tagPatternCount} {tagPatternCount === 1 ? "pattern" : "patterns"}
</span>
)}
</div>
}
/>
)}
<SwitchRow
icon={HardDrive}
label="Git LFS objects"
@@ -1004,6 +1085,24 @@ export function GitHubMirrorSettings({
</CardSection>
</SettingsCard>
)}
<ReleaseTagRulesDialog
open={tagRulesOpen}
onOpenChange={setTagRulesOpen}
includeTags={mirrorOptions.releaseIncludeTags ?? []}
excludeTags={mirrorOptions.releaseExcludeTags ?? []}
previewRepositories={globalPreviewRepos}
releaseType={mirrorOptions.releaseType}
onSave={(includeTags, excludeTags) => {
// Both fields in one call: handleMirrorChange would spread the same
// stale mirrorOptions twice and drop the first list.
onMirrorOptionsChange({
...mirrorOptions,
releaseIncludeTags: includeTags,
releaseExcludeTags: excludeTags,
});
}}
/>
</>
);
}
+466 -140
View File
@@ -1,4 +1,4 @@
import { Fragment, useEffect, useMemo, useState } from "react";
import { useEffect, useMemo, useState } from "react";
import {
Dialog,
DialogContent,
@@ -17,17 +17,24 @@ import {
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import type { MirrorOverrides } from "@/lib/db/schema";
import { ChevronRight } from "lucide-react";
import { cn } from "@/lib/utils";
import { withBase } from "@/lib/base-path";
import type { MirrorOverrides, ReleaseType } from "@/lib/db/schema";
import {
compileReleaseTagPattern,
getMirrorOverrideGating,
MIRROR_OVERRIDE_LABELS,
normalizeReleaseAssetLimit,
normalizeReleaseLimit,
normalizeReleaseTagPatterns,
normalizeReleaseType,
UI_MIRROR_OVERRIDE_KEYS,
type InheritedMirrorOptions,
type MirrorOverrideKey,
type OrgSelectionOverrideKey,
} from "@/lib/utils/mirror-overrides";
import { ReleaseTagRulesDialog } from "./ReleaseTagRulesDialog";
/**
* Tri-state value for one flag in the dialog. "inherit" persists as absent,
@@ -35,6 +42,25 @@ import {
*/
type TriState = "inherit" | "on" | "off";
/**
* The release type is a choice rather than a flag, so it gets its own three
* values: inherit plus the two kinds. "inherit" persists as absent.
*/
type ReleaseTypeChoice = "inherit" | ReleaseType;
/**
* What the single Releases row offers. It covers both stored fields at once:
* "off" pins the `mirrorReleases` flag false, the two kinds pin it true
* together with a `releaseType`, and "inherit" pins neither.
*/
type ReleaseChoice = "inherit" | ReleaseType | "off";
/** How the value the row inherits is described in the hint beside its label. */
const RELEASE_TYPE_LABELS: Record<ReleaseType, string> = {
all: "all releases",
stable: "stable releases only",
};
type DialogOverrideKey = MirrorOverrideKey | OrgSelectionOverrideKey;
function toTriState(value: boolean | null | undefined): TriState {
@@ -67,6 +93,45 @@ function parseReleaseAssetLimitDraft(text: string): number | undefined {
return normalizeReleaseAssetLimit(Number(trimmed));
}
type TagPatternKey = "releaseIncludeTags" | "releaseExcludeTags";
/**
* The stored list as the comma-separated text the box edits. An absent or
* unusable list reads as empty, which is what "inherit" means here: an empty
* list is not a pin, so a repository cannot clear a pattern its organization
* set.
*/
function buildReleaseTagDraft(
value: MirrorOverrides | null | undefined,
key: TagPatternKey
): string {
return (normalizeReleaseTagPatterns(value?.[key]) ?? []).join(", ");
}
/**
* Read one tag-pattern box.
*
* `patterns` is what gets stored, or undefined when the box means "inherit".
* `unusable` names the entries that would not take effect - too long, or a
* regular expression that does not compile - so a typo is visible before saving
* rather than silently ignored during a mirror run.
*/
function readTagPatternDraft(text: string): {
patterns: string[] | undefined;
unusable: string[];
} {
const entries = text
.split(",")
.map((entry) => entry.trim())
.filter(Boolean);
const patterns = normalizeReleaseTagPatterns(entries) ?? [];
const kept = new Set(patterns.map((pattern) => pattern.toLowerCase()));
const unusable = entries.filter(
(entry) => !kept.has(entry.toLowerCase()) || compileReleaseTagPattern(entry) === null
);
return { patterns: patterns.length > 0 ? patterns : undefined, unusable };
}
export interface MirrorOverridesDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
@@ -74,6 +139,12 @@ export interface MirrorOverridesDialogProps {
targetName: string;
/** "repository" or "organization", used for wording and gating rules. */
targetKind: "repository" | "organization";
/**
* Database id of the repository being edited. Only a single repository has one,
* and only with it can the tag-rule dialog show which published releases its
* rules keep or delete.
*/
targetId?: string | null;
value?: MirrorOverrides | null;
/**
* Effective values from the tiers above this one, used to label what
@@ -100,6 +171,7 @@ export function MirrorOverridesDialog({
onOpenChange,
targetName,
targetKind,
targetId,
value,
inheritedFrom,
inheritedLabel = "global settings",
@@ -128,7 +200,22 @@ export function MirrorOverridesDialog({
const [releaseAssetLimitDraft, setReleaseAssetLimitDraft] = useState(() =>
buildReleaseAssetLimitDraft(value)
);
const [releaseTypeDraft, setReleaseTypeDraft] = useState<ReleaseTypeChoice>(
() => buildReleaseTypeDraft(value)
);
const [includeTagsDraft, setIncludeTagsDraft] = useState(() =>
buildReleaseTagDraft(value, "releaseIncludeTags")
);
const [excludeTagsDraft, setExcludeTagsDraft] = useState(() =>
buildReleaseTagDraft(value, "releaseExcludeTags")
);
const [isSaving, setIsSaving] = useState(false);
const [tagRulesOpen, setTagRulesOpen] = useState(false);
// The repositories an organization override covers. Its tag-rule preview needs
// one chosen repository, so the rule dialog offers these in a picker.
const [orgPreviewRepos, setOrgPreviewRepos] = useState<
{ id: string; label: string }[]
>([]);
// Re-seed whenever the dialog opens or targets a different object, so a
// previous edit never leaks into the next one.
@@ -137,9 +224,49 @@ export function MirrorOverridesDialog({
setDraft(buildDraft(value, dialogKeys));
setReleaseLimitDraft(buildReleaseLimitDraft(value));
setReleaseAssetLimitDraft(buildReleaseAssetLimitDraft(value));
setReleaseTypeDraft(buildReleaseTypeDraft(value));
setIncludeTagsDraft(buildReleaseTagDraft(value, "releaseIncludeTags"));
setExcludeTagsDraft(buildReleaseTagDraft(value, "releaseExcludeTags"));
}
}, [open, value, dialogKeys]);
// An organization override fans out over every repository it mirrors, so its
// tag-rule preview cannot name a single destination up front. Fetch the
// repositories it covers per open/target; the releases for whichever one the
// user then picks come from the repository releases route.
useEffect(() => {
if (!open || targetKind !== "organization" || !targetId) {
setOrgPreviewRepos([]);
return;
}
const controller = new AbortController();
void (async () => {
try {
const response = await fetch(
withBase(`/api/organizations/${targetId}/preview-repositories`),
{
signal: controller.signal,
credentials: "include",
headers: { Accept: "application/json" },
}
);
const payload = await response.json().catch(() => null);
const list = Array.isArray(payload?.repositories) ? payload.repositories : [];
setOrgPreviewRepos(
list
.filter((entry: { id?: unknown }) => typeof entry.id === "string")
.map((entry: { id: string; label?: unknown }) => ({
id: entry.id,
label: typeof entry.label === "string" ? entry.label : entry.id,
}))
);
} catch {
if (!controller.signal.aborted) setOrgPreviewRepos([]);
}
})();
return () => controller.abort();
}, [open, targetKind, targetId]);
// What each flag currently resolves to, including the in-progress edit.
// Drives the labels and release-limit gates so they react live as issues
// and releases are toggled.
@@ -166,6 +293,18 @@ export function MirrorOverridesDialog({
[targetKind, isStarred, starredCodeOnly, effective, destinationProvider]
);
// What the tag-rule dialog previews. A repository override has the one row it
// edits; an organization override has the list its picker offers, and nothing
// until that list arrives. The global settings page never reaches this dialog.
const previewRepositories = useMemo<
readonly { id: string; label: string }[] | null
>(() => {
if (targetKind === "repository") {
return targetId ? [{ id: targetId, label: targetName }] : null;
}
return orgPreviewRepos.length > 0 ? orgPreviewRepos : null;
}, [targetKind, targetId, targetName, orgPreviewRepos]);
const pinnedReleaseLimit = parseReleaseLimitDraft(releaseLimitDraft);
// Non-empty text that does not parse to a usable limit (0, negative, junk).
const releaseLimitInvalid =
@@ -175,13 +314,52 @@ export function MirrorOverridesDialog({
const releaseAssetLimitInvalid =
releaseAssetLimitDraft.trim() !== "" && pinnedReleaseAssetLimit === undefined;
const overriddenCount = useMemo(
() =>
Object.values(draft).filter((state) => state !== "inherit").length +
(pinnedReleaseLimit !== undefined ? 1 : 0) +
(pinnedReleaseAssetLimit !== undefined ? 1 : 0),
[draft, pinnedReleaseLimit, pinnedReleaseAssetLimit]
);
const pinnedReleaseType: ReleaseType | undefined =
releaseTypeDraft === "inherit" ? undefined : normalizeReleaseType(releaseTypeDraft);
// One control drives two stored fields. A stored "releases on" with no kind
// pinned still reads as Inherit and keeps its pin on save, so an override set
// before the kind filter existed is never dropped by merely opening the dialog.
const releaseChoice: ReleaseChoice =
draft.mirrorReleases === "off"
? "off"
: releaseTypeDraft !== "inherit"
? releaseTypeDraft
: "inherit";
const applyReleaseChoice = (next: ReleaseChoice) => {
setDraft((prev) => ({
...prev,
mirrorReleases:
next === "inherit" ? "inherit" : next === "off" ? "off" : "on",
}));
// "off" leaves the pinned kind alone, so switching releases back on goes
// straight back to the kind that was set before.
if (next !== "off") setReleaseTypeDraft(next);
};
const includeTags = readTagPatternDraft(includeTagsDraft);
const excludeTags = readTagPatternDraft(excludeTagsDraft);
const includeTagsInvalid = includeTags.unusable.length > 0;
const excludeTagsInvalid = excludeTags.unusable.length > 0;
const tagPatternsInvalid = includeTagsInvalid || excludeTagsInvalid;
const unusableTagPatterns = [...includeTags.unusable, ...excludeTags.unusable];
const tagPatternCount =
(includeTags.patterns?.length ?? 0) + (excludeTags.patterns?.length ?? 0);
// How many rows pin a value, which is what "Reset all" waits on. The Releases
// row counts once even though it writes two fields.
const overriddenCount =
Object.entries(draft).filter(
([key, state]) => key !== "mirrorReleases" && state !== "inherit"
).length +
(draft.mirrorReleases !== "inherit" || releaseTypeDraft !== "inherit"
? 1
: 0) +
(pinnedReleaseLimit !== undefined ? 1 : 0) +
(pinnedReleaseAssetLimit !== undefined ? 1 : 0) +
(includeTags.patterns !== undefined ? 1 : 0) +
(excludeTags.patterns !== undefined ? 1 : 0);
const handleSave = async () => {
setIsSaving(true);
@@ -201,6 +379,13 @@ export function MirrorOverridesDialog({
if (pinnedReleaseAssetLimit !== undefined) {
next.releaseAssetLimit = pinnedReleaseAssetLimit;
}
if (pinnedReleaseType !== undefined) next.releaseType = pinnedReleaseType;
if (includeTags.patterns !== undefined) {
next.releaseIncludeTags = includeTags.patterns;
}
if (excludeTags.patterns !== undefined) {
next.releaseExcludeTags = excludeTags.patterns;
}
await onSave(Object.keys(next).length > 0 ? next : null);
onOpenChange(false);
} finally {
@@ -212,6 +397,9 @@ export function MirrorOverridesDialog({
setDraft(buildDraft(null, dialogKeys));
setReleaseLimitDraft("");
setReleaseAssetLimitDraft("");
setReleaseTypeDraft("inherit");
setIncludeTagsDraft("");
setExcludeTagsDraft("");
};
const releaseLimitReason = gating.releaseLimit;
@@ -235,6 +423,52 @@ export function MirrorOverridesDialog({
releaseAssetLimitDraft.trim() === "" &&
inheritedReleaseAssetLimit !== undefined;
// The Releases row is only locked by a tier-level clamp: the starred setting,
// a push destination, or a source with no releases. Choosing "Off" here must
// not disable the row, or the choice could never be undone.
const releasesRowReason = gating.mirrorReleases;
const releasesRowDisabled = !!releasesRowReason;
// What the row resolves to while it inherits: the kind the tiers above pin,
// which the runtime defaults to "all" when nothing names one. The tag-rule
// preview needs it, because the retention pass applies the type filter and
// the tag patterns together when it decides what stays in the destination.
const effectiveReleaseType: ReleaseType =
pinnedReleaseType ?? inheritedFrom?.releaseType ?? "all";
const showReleasesHint =
!inheritedLoading && !releasesRowDisabled && releaseChoice === "inherit";
const releasesHintText = effective.mirrorReleases
? RELEASE_TYPE_LABELS[effectiveReleaseType]
: "off";
// Both boxes are gated by the same condition (they narrow the release filter),
// so one reason covers the pair.
const releaseTagsReason = gating.releaseIncludeTags ?? gating.releaseExcludeTags;
const releaseTagsDisabled = !!releaseTagsReason;
// An empty inherited list is no filter, so it is not worth a hint; only a list
// that is actually filtering gets named beside the empty box.
const inheritedIncludeTagText =
inheritedFrom?.releaseIncludeTags && inheritedFrom.releaseIncludeTags.length > 0
? inheritedFrom.releaseIncludeTags.join(", ")
: undefined;
const inheritedExcludeTagText =
inheritedFrom?.releaseExcludeTags && inheritedFrom.releaseExcludeTags.length > 0
? inheritedFrom.releaseExcludeTags.join(", ")
: undefined;
// The two lists are edited together in one dialog, so the inherited rules are
// named as a single hint: "only v*, except *-rc.*". An empty inherited list is
// no filter, so it is not worth naming.
const inheritedTagText = [
inheritedIncludeTagText ? `only ${inheritedIncludeTagText}` : null,
inheritedExcludeTagText ? `except ${inheritedExcludeTagText}` : null,
]
.filter((text): text is string => !!text)
.join(", ");
const showTagsHint =
!inheritedLoading &&
!releaseTagsDisabled &&
tagPatternCount === 0 &&
inheritedTagText !== "";
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="sm:max-w-md">
@@ -247,21 +481,195 @@ export function MirrorOverridesDialog({
</DialogDescription>
</DialogHeader>
<div className="space-y-3 py-2">
{dialogKeys.map((key) => {
const disabledReason = gating[key];
const isDisabled = !!disabledReason;
const inherited =
key === "skipForks" ? inheritedFrom?.skipForks : inheritedFrom?.[key];
const showHint =
!inheritedLoading &&
!isDisabled &&
draft[key] === "inherit" &&
inherited !== undefined;
<div className="max-h-[62vh] overflow-y-auto pr-1">
<div className="space-y-3">
{dialogKeys.map((key) => {
// Everything release related is grouped under the Releases row,
// so the limits and the tag rules never drift away from the
// filter they narrow.
if (key === "mirrorReleases") {
return (
<div key={key} className="space-y-3">
<div className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor="override-releases"
className={
releasesRowDisabled
? "flex-1 text-muted-foreground"
: "flex-1"
}
>
{MIRROR_OVERRIDE_LABELS.mirrorReleases}
{showReleasesHint && (
<span className="ml-2 text-xs font-normal text-muted-foreground">
currently {releasesHintText}
</span>
)}
</Label>
<Select
value={releaseChoice}
disabled={releasesRowDisabled}
onValueChange={(next) =>
applyReleaseChoice(next as ReleaseChoice)
}
>
<SelectTrigger id="override-releases" className="w-36">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="inherit">Inherit</SelectItem>
<SelectItem value="all">All</SelectItem>
<SelectItem value="stable">Stable only</SelectItem>
<SelectItem value="off">Off</SelectItem>
</SelectContent>
</Select>
</div>
{releasesRowReason && (
<p className="text-xs text-muted-foreground">
{releasesRowReason}
</p>
)}
</div>
return (
<Fragment key={key}>
<div className="space-y-1">
<div className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor="override-releaseLimit"
className={
releaseLimitDisabled
? "flex-1 text-muted-foreground"
: "flex-1"
}
>
{MIRROR_OVERRIDE_LABELS.releaseLimit}
{showReleaseLimitHint && (
<span className="ml-2 text-xs font-normal text-muted-foreground">
currently {inheritedReleaseLimit}
</span>
)}
</Label>
<Input
id="override-releaseLimit"
type="number"
inputMode="numeric"
min={1}
step={1}
placeholder="Inherit"
className="w-32"
value={releaseLimitDraft}
disabled={releaseLimitDisabled}
aria-invalid={releaseLimitInvalid || undefined}
onChange={(event) =>
setReleaseLimitDraft(event.target.value)
}
/>
</div>
{!releaseLimitDisabled && releaseLimitInvalid && (
<p className="text-xs text-destructive">
Enter a whole number of 1 or more, or leave it empty to
inherit.
</p>
)}
</div>
<div className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor="override-releaseAssetLimit"
className={
releaseAssetLimitDisabled
? "flex-1 text-muted-foreground"
: "flex-1"
}
>
{MIRROR_OVERRIDE_LABELS.releaseAssetLimit}
{showReleaseAssetLimitHint && (
<span className="ml-2 text-xs font-normal text-muted-foreground">
currently{" "}
{inheritedReleaseAssetLimit === null
? "all"
: inheritedReleaseAssetLimit}
</span>
)}
</Label>
<Input
id="override-releaseAssetLimit"
type="number"
inputMode="numeric"
min={0}
step={1}
placeholder="Inherit"
className="w-32"
value={releaseAssetLimitDraft}
disabled={releaseAssetLimitDisabled}
aria-invalid={releaseAssetLimitInvalid || undefined}
onChange={(event) =>
setReleaseAssetLimitDraft(event.target.value)
}
/>
</div>
{!releaseAssetLimitDisabled && releaseAssetLimitInvalid && (
<p className="text-xs text-destructive">
Enter a whole number of 0 or more, or leave it empty to
inherit.
</p>
)}
</div>
{/* Tag patterns narrow the kind filter further, for upstreams
that never set the pre-release flag but tag v1.2.3-rc.1. */}
<div className="space-y-1">
<button
id="override-tag-rules"
type="button"
className={cn(
"flex w-full items-center gap-2 rounded-md border border-dashed px-3 py-2 text-left transition-colors hover:bg-muted",
releaseTagsDisabled && "cursor-not-allowed opacity-60"
)}
disabled={releaseTagsDisabled}
onClick={() => setTagRulesOpen(true)}
>
<ChevronRight className="h-4 w-4 shrink-0 text-muted-foreground" />
<span className="text-sm font-medium">Custom tag rules</span>
<span className="text-xs text-muted-foreground">
optional
</span>
<span className="ml-auto text-xs text-muted-foreground">
{tagPatternCount > 0
? `${tagPatternCount} set`
: showTagsHint
? `currently ${inheritedTagText}`
: "none set"}
</span>
</button>
{releaseTagsReason ? (
<p className="text-xs text-muted-foreground">
{releaseTagsReason}
</p>
) : tagPatternsInvalid ? (
<p className="text-xs text-destructive">
{unusableTagPatterns.join(", ")} cannot be used as a tag
pattern.
</p>
) : null}
</div>
</div>
);
}
const disabledReason = gating[key];
const isDisabled = !!disabledReason;
const inherited =
key === "skipForks" ? inheritedFrom?.skipForks : inheritedFrom?.[key];
const showHint =
!inheritedLoading &&
!isDisabled &&
draft[key] === "inherit" &&
inherited !== undefined;
return (
<div key={key} className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor={`override-${key}`}
@@ -297,125 +705,15 @@ export function MirrorOverridesDialog({
</p>
)}
</div>
);
})}
{/* The limits only mean something while releases are mirrored,
so they sit directly under that row. */}
{key === "mirrorReleases" && (
<>
<div className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor="override-releaseLimit"
className={
releaseLimitDisabled
? "flex-1 text-muted-foreground"
: "flex-1"
}
>
{MIRROR_OVERRIDE_LABELS.releaseLimit}
{showReleaseLimitHint && (
<span className="ml-2 text-xs font-normal text-muted-foreground">
currently {inheritedReleaseLimit}
</span>
)}
</Label>
<Input
id="override-releaseLimit"
type="number"
inputMode="numeric"
min={1}
step={1}
placeholder="Inherit"
className="w-32"
value={releaseLimitDraft}
disabled={releaseLimitDisabled}
aria-invalid={releaseLimitInvalid || undefined}
onChange={(event) => setReleaseLimitDraft(event.target.value)}
/>
</div>
{releaseLimitDisabled ? (
<p className="text-xs text-muted-foreground">
{releaseLimitReason}
</p>
) : releaseLimitInvalid ? (
<p className="text-xs text-destructive">
Enter a whole number of 1 or more, or leave it empty to
inherit.
</p>
) : (
<p className="text-xs text-muted-foreground">
Newest releases to keep. Older ones past the limit are
removed from Gitea on the next sync.
</p>
)}
</div>
<div className="space-y-1">
<div className="flex items-center justify-between gap-4">
<Label
htmlFor="override-releaseAssetLimit"
className={
releaseAssetLimitDisabled
? "flex-1 text-muted-foreground"
: "flex-1"
}
>
{MIRROR_OVERRIDE_LABELS.releaseAssetLimit}
{showReleaseAssetLimitHint && (
<span className="ml-2 text-xs font-normal text-muted-foreground">
currently{" "}
{inheritedReleaseAssetLimit === null
? "all"
: inheritedReleaseAssetLimit}
</span>
)}
</Label>
<Input
id="override-releaseAssetLimit"
type="number"
inputMode="numeric"
min={0}
step={1}
placeholder="Inherit"
className="w-32"
value={releaseAssetLimitDraft}
disabled={releaseAssetLimitDisabled}
aria-invalid={releaseAssetLimitInvalid || undefined}
onChange={(event) =>
setReleaseAssetLimitDraft(event.target.value)
}
/>
</div>
{releaseAssetLimitDisabled ? (
<p className="text-xs text-muted-foreground">
{releaseAssetLimitReason}
</p>
) : releaseAssetLimitInvalid ? (
<p className="text-xs text-destructive">
Enter a whole number of 0 or more, or leave it empty to
inherit.
</p>
) : (
<p className="text-xs text-muted-foreground">
Assets are uploaded only for this many of the newest
releases; 0 keeps release notes only. Assets already in
Gitea are never removed.
</p>
)}
</div>
</>
)}
</Fragment>
);
})}
<p className="text-xs text-muted-foreground">
Inherit uses the {inheritedLabel}.
</p>
</div>
</div>
<p className="text-xs text-muted-foreground">
Inherit uses the {inheritedLabel}. Git LFS is worth turning off for
repositories whose LFS fetch fails, since Gitea aborts the whole
migration when it cannot pull them.
</p>
<DialogFooter className="gap-2 sm:gap-0">
<Button
variant="ghost"
@@ -433,11 +731,32 @@ export function MirrorOverridesDialog({
</Button>
<Button
onClick={handleSave}
disabled={isSaving || releaseLimitInvalid || releaseAssetLimitInvalid}
disabled={
isSaving ||
releaseLimitInvalid ||
releaseAssetLimitInvalid ||
includeTagsInvalid ||
excludeTagsInvalid
}
>
{isSaving ? "Saving..." : "Save"}
</Button>
</DialogFooter>
{/* Nested on purpose: the pattern syntax needs more room than this
compact override dialog has. */}
<ReleaseTagRulesDialog
open={tagRulesOpen}
onOpenChange={setTagRulesOpen}
includeTags={includeTags.patterns ?? []}
excludeTags={excludeTags.patterns ?? []}
previewRepositories={previewRepositories}
releaseType={effectiveReleaseType}
onSave={(include, exclude) => {
setIncludeTagsDraft(include.join(", "));
setExcludeTagsDraft(exclude.join(", "));
}}
/>
</DialogContent>
</Dialog>
);
@@ -467,3 +786,10 @@ function buildReleaseAssetLimitDraft(
const limit = normalizeReleaseAssetLimit(value?.releaseAssetLimit);
return limit === undefined ? "" : String(limit);
}
/** An unusable stored value reads as inherit, matching how the resolver treats it. */
function buildReleaseTypeDraft(
value: MirrorOverrides | null | undefined
): ReleaseTypeChoice {
return normalizeReleaseType(value?.releaseType) ?? "inherit";
}
@@ -0,0 +1,380 @@
import { useEffect, useState } from "react";
import { Check, XCircle } from "lucide-react";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { cn } from "@/lib/utils";
import {
compileReleaseTagPattern,
compileReleaseTagPatterns,
MAX_RELEASE_TAG_PATTERNS,
MAX_RELEASE_TAG_PATTERN_LENGTH,
normalizeReleaseTagPatterns,
releaseMatchesTagFilters,
releaseMatchesType,
splitReleaseTagEntries,
} from "@/lib/utils/mirror-overrides";
import type { ReleaseType } from "@/lib/db/schema";
import { withBase } from "@/lib/base-path";
/** A release the destination already holds for the repository being edited. */
type PublishedRelease = {
tag: string;
prerelease: boolean | null;
};
/**
* One repository the rule can be previewed against. A repository override offers
* exactly one; an organization override offers every repository it covers, and
* the picker lets the user choose which one the preview runs against.
*/
type PreviewRepository = {
id: string;
label: string;
};
type PreviewState =
| { status: "loading" }
| { status: "error"; error: string }
| {
status: "ready";
destination: string;
releases: PublishedRelease[];
truncated: boolean;
};
/**
* One tag-rule box read as text.
*
* `patterns` is what gets stored. `unusable` names the entries that would never
* take effect - too long, past the count cap, or a regular expression that does
* not compile - so a typo is visible here instead of being silently dropped
* during a mirror run.
*/
function readTagRuleDraft(text: string): { patterns: string[]; unusable: string[] } {
// Commas inside /.../ belong to the regex, not to the list.
const entries = splitReleaseTagEntries(text);
const patterns = normalizeReleaseTagPatterns(entries) ?? [];
const kept = new Set(patterns.map((pattern) => pattern.toLowerCase()));
const unusable = entries.filter(
(entry) => !kept.has(entry.toLowerCase()) || compileReleaseTagPattern(entry) === null
);
return { patterns, unusable };
}
export interface ReleaseTagRulesDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
includeTags?: readonly string[] | null;
excludeTags?: readonly string[] | null;
/** An empty list means that side does not filter. */
onSave: (includeTags: string[], excludeTags: string[]) => void;
/**
* Repositories whose published releases the draft rules can be checked
* against. When set, the dialog lists what the destination already holds and
* marks the releases a save keeps and the ones the next sync would delete. A
* repository override passes the single repository it edits; an organization
* override passes every repository it covers and the dialog adds a picker so
* the user previews one of them. It is left out where nothing single applies:
* the global settings cover every repository at once.
*/
previewRepositories?: readonly PreviewRepository[] | null;
/**
* The release type this repository ends up with. The retention pass applies the
* type filter and the tag patterns together, so the preview would be wrong if
* it only looked at the patterns. Defaults to "all" (no type filter).
*/
releaseType?: ReleaseType;
}
/**
* The tag patterns that narrow which releases are mirrored. They live in a
* dialog rather than inline on the settings page because the syntax needs room
* to explain itself; the page only shows how many rules are set.
*/
export function ReleaseTagRulesDialog({
open,
onOpenChange,
includeTags,
excludeTags,
onSave,
previewRepositories,
releaseType = "all",
}: ReleaseTagRulesDialogProps) {
const [includeText, setIncludeText] = useState("");
const [excludeText, setExcludeText] = useState("");
const [preview, setPreview] = useState<PreviewState | null>(null);
// Which repository the preview runs against. An organization override offers
// several; a repository override has only the one it edits.
const [selectedRepoId, setSelectedRepoId] = useState<string | null>(null);
// Re-seeded on every open so a config change elsewhere cannot be overwritten
// by a draft left over from a previous visit.
useEffect(() => {
if (!open) return;
setIncludeText((normalizeReleaseTagPatterns(includeTags) ?? []).join(", "));
setExcludeText((normalizeReleaseTagPatterns(excludeTags) ?? []).join(", "));
}, [open, includeTags, excludeTags]);
// The published releases are read once per open. Judging them against the rules
// happens in the browser, so editing a pattern afterwards costs no request.
useEffect(() => {
if (!open || !selectedRepoId) {
setPreview(null);
return;
}
const controller = new AbortController();
setPreview({ status: "loading" });
void (async () => {
try {
const response = await fetch(
withBase(`/api/repositories/${selectedRepoId}/releases`),
{
signal: controller.signal,
credentials: "include",
headers: { Accept: "application/json" },
}
);
const payload = await response.json().catch(() => null);
const data = payload?.data ?? payload;
if (!response.ok || data?.success === false) {
setPreview({
status: "error",
error:
typeof data?.error === "string" ? data.error : `Request failed (${response.status})`,
});
return;
}
setPreview({
status: "ready",
destination: typeof data?.destination === "string" ? data.destination : "",
releases: Array.isArray(data?.releases) ? data.releases : [],
truncated: data?.truncated === true,
});
} catch (error) {
if (controller.signal.aborted) return;
setPreview({
status: "error",
error: error instanceof Error ? error.message : "The destination could not be reached",
});
}
})();
return () => controller.abort();
}, [open, selectedRepoId]);
// Keep the selection pointed at a repository that is actually offered. Opening
// a different organization, or one whose repository list changed, drops a
// stale selection back to the first entry.
useEffect(() => {
if (!open) return;
const offered = previewRepositories ?? [];
setSelectedRepoId((previous) =>
previous && offered.some((repository) => repository.id === previous)
? previous
: (offered[0]?.id ?? null)
);
}, [open, previewRepositories]);
const include = readTagRuleDraft(includeText);
const exclude = readTagRuleDraft(excludeText);
const unusable = [
...include.unusable.map((pattern) => `Include: ${pattern}`),
...exclude.unusable.map((pattern) => `Exclude: ${pattern}`),
];
// Re-run what the retention pass does to the destination: a published release
// stays when its tag passes the patterns and - only where the destination
// reported a pre-release flag - the type filter as well. An unknown flag is
// never guessed at, exactly as in a mirror run.
const kept: string[] = [];
const removed: string[] = [];
if (preview?.status === "ready") {
const includePatterns = compileReleaseTagPatterns(include.patterns);
const excludePatterns = compileReleaseTagPatterns(exclude.patterns);
for (const release of preview.releases) {
const survives =
releaseMatchesTagFilters(release.tag, includePatterns, excludePatterns) &&
(releaseType === "all" ||
typeof release.prerelease !== "boolean" ||
releaseMatchesType({ prerelease: release.prerelease }, releaseType));
(survives ? kept : removed).push(release.tag);
}
}
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="sm:max-w-lg">
<DialogHeader>
<DialogTitle>Custom release tag rules</DialogTitle>
<DialogDescription>
Narrow which releases are mirrored by their tag name. An excluded tag
wins when both lists name it.
</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div className="space-y-1">
<Label htmlFor="tag-rules-include">Include tags</Label>
<Input
id="tag-rules-include"
type="text"
placeholder="v*, release-*"
value={includeText}
onChange={(e) => setIncludeText(e.target.value)}
/>
<p className="text-xs text-muted-foreground">
When set, only releases whose tag matches one of these are mirrored.
Empty means no include filter.
</p>
</div>
<div className="space-y-1">
<Label htmlFor="tag-rules-exclude">Exclude tags</Label>
<Input
id="tag-rules-exclude"
type="text"
placeholder="*-rc.*, *-alpha.*, *-next.*"
value={excludeText}
onChange={(e) => setExcludeText(e.target.value)}
/>
<p className="text-xs text-muted-foreground">
Releases whose tag matches one of these are skipped, even when an
include pattern matches. Empty means no exclude filter.
</p>
</div>
<p className="text-xs text-muted-foreground">
Patterns are matched against the whole tag name, ignoring case:{" "}
<code className="rounded bg-muted px-1 py-0.5">v1.*</code> matches v1.4
and v1.4.0, <code className="rounded bg-muted px-1 py-0.5">?</code> is
exactly one character, and{" "}
<code className="rounded bg-muted px-1 py-0.5">/^\d+\.\d+$/</code> is a
regular expression. Up to {MAX_RELEASE_TAG_PATTERNS} patterns of{" "}
{MAX_RELEASE_TAG_PATTERN_LENGTH} characters each. Releases already in
Gitea that stop matching are removed on the next sync.
</p>
{previewRepositories && previewRepositories.length > 0 ? (
<div className="space-y-2 rounded-md border p-3">
{previewRepositories.length > 1 ? (
<div className="space-y-1">
<Label htmlFor="tag-rules-preview-repo">Preview repository</Label>
<Select
value={selectedRepoId ?? ""}
onValueChange={(next) => setSelectedRepoId(next)}
>
<SelectTrigger id="tag-rules-preview-repo" className="w-full">
<SelectValue placeholder="Choose a repository" />
</SelectTrigger>
<SelectContent>
{previewRepositories.map((repository) => (
<SelectItem key={repository.id} value={repository.id}>
{repository.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
) : null}
<div className="flex items-center justify-between gap-2">
<span className="text-sm font-medium">Already in Gitea</span>
{preview?.status === "ready" && preview.releases.length > 0 ? (
<span
className={cn(
"text-xs",
removed.length > 0 ? "text-destructive" : "text-muted-foreground"
)}
>
{removed.length === 0
? `all ${kept.length} kept`
: `${removed.length} of ${preview.releases.length} would be removed`}
</span>
) : null}
</div>
{preview?.status === "loading" ? (
<p className="text-xs text-muted-foreground">Checking the destination…</p>
) : null}
{preview?.status === "error" ? (
<p className="text-xs text-destructive">{preview.error}</p>
) : null}
{preview?.status === "ready" && preview.releases.length === 0 ? (
<p className="text-xs text-muted-foreground">
{preview.destination
? `Nothing is published in ${preview.destination} yet.`
: "Nothing is published there yet."}
</p>
) : null}
{preview?.status === "ready" && preview.releases.length > 0 ? (
<>
<div className="grid max-h-40 gap-x-3 gap-y-1 overflow-y-auto pr-1 sm:grid-cols-2">
{removed.map((tag) => (
<div key={tag} className="flex items-center gap-2 text-xs">
<XCircle className="h-3.5 w-3.5 shrink-0 text-destructive" />
<span className="truncate font-mono text-muted-foreground line-through">
{tag}
</span>
</div>
))}
{kept.map((tag) => (
<div key={tag} className="flex items-center gap-2 text-xs">
<Check className="h-3.5 w-3.5 shrink-0 text-green-600 dark:text-green-400" />
<span className="truncate font-mono">{tag}</span>
</div>
))}
</div>
<p className="text-xs text-muted-foreground">
The destination is checked against these rules on every sync, so a
crossed-out release is deleted the next time this repository syncs.
{preview.truncated
? ` Only the newest ${preview.releases.length} published releases are listed.`
: ""}
</p>
</>
) : null}
</div>
) : null}
{unusable.length > 0 && (
<p className="rounded-md bg-destructive/10 px-3 py-2 text-xs text-destructive">
Cannot be used: {unusable.join(", ")}
</p>
)}
</div>
<DialogFooter>
<Button variant="outline" onClick={() => onOpenChange(false)}>
Cancel
</Button>
<Button
onClick={() => {
onSave(include.patterns, exclude.patterns);
onOpenChange(false);
}}
disabled={unusable.length > 0}
>
Save rules
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
+14 -1
View File
@@ -121,6 +121,10 @@ interface OptionRowProps {
badge?: string;
/** Control rendered on the right (switch, pill, input...). */
right?: ReactNode;
/** Rendered under the description, e.g. a link that opens a sub-dialog. */
subContent?: ReactNode;
/** Rows whose control is a stack of fields (two inputs) sit top-aligned. */
align?: "center" | "start";
className?: string;
}
@@ -131,10 +135,18 @@ export function OptionRow({
info,
badge,
right,
subContent,
align,
className,
}: OptionRowProps) {
return (
<div className={cn("flex items-center gap-4", className)}>
<div
className={cn(
"flex gap-4",
align === "start" ? "items-start" : "items-center",
className
)}
>
<div className="flex-1 min-w-0 space-y-1">
<div className="flex items-center gap-2">
{Icon && <Icon className="h-3.5 w-3.5 text-muted-foreground" />}
@@ -149,6 +161,7 @@ export function OptionRow({
{description && (
<p className="text-[13px] text-muted-foreground">{description}</p>
)}
{subContent}
</div>
{right}
</div>
+5 -2
View File
@@ -78,11 +78,14 @@ export function Dashboard() {
// Create a stable callback using useCallback
const handleNewMessage = useCallback((data: MirrorJob) => {
// A "warning" entry is an activity alert, never a repository or organization status.
const status = data.status;
if (status === "warning") return;
if (data.repositoryId) {
setRepositories((prevRepos) =>
prevRepos.map((repo) =>
repo.id === data.repositoryId
? { ...repo, status: data.status, details: data.details }
? { ...repo, status, details: data.details }
: repo
)
);
@@ -90,7 +93,7 @@ export function Dashboard() {
setOrganizations((prevOrgs) =>
prevOrgs.map((org) =>
org.id === data.organizationId
? { ...org, status: data.status, details: data.details }
? { ...org, status, details: data.details }
: org
)
);
@@ -29,6 +29,8 @@ function activityIcon(status: MirrorJob["status"]) {
return { Icon: Sparkles, color: "text-muted-foreground" };
case "failed":
return { Icon: TriangleAlert, color: "text-red-500" };
case "warning":
return { Icon: TriangleAlert, color: "text-amber-500" };
default:
return { Icon: Activity, color: "text-muted-foreground" };
}
@@ -268,6 +268,7 @@ export function OrganizationList({
}}
targetKind="organization"
targetName={overridesTarget?.name ?? ""}
targetId={overridesTarget?.id ?? null}
value={overridesTarget?.mirrorOverrides ?? null}
destinationProvider={giteaConfig?.provider}
inheritedFrom={{
+4 -1
View File
@@ -101,11 +101,14 @@ export default function Repository() {
// Create a stable callback using useCallback
const handleNewMessage = useCallback((data: MirrorJob) => {
// A "warning" entry is an activity alert, never a repository status.
const status = data.status;
if (status === "warning") return;
if (data.repositoryId) {
setRepositories((prevRepos) =>
prevRepos.map((repo) =>
repo.id === data.repositoryId
? { ...repo, status: data.status, details: data.details }
? { ...repo, status, details: data.details }
: repo
)
);
@@ -37,6 +37,8 @@ import {
mirrorOptionsToFlags,
normalizeReleaseAssetLimit,
normalizeReleaseLimit,
normalizeReleaseTagPatterns,
normalizeReleaseType,
type MirrorOverrideKey,
} from "@/lib/utils/mirror-overrides";
import { Card, CardContent } from "@/components/ui/card";
@@ -189,6 +191,12 @@ export default function RepositoryTable({
if (orgReleaseLimit !== undefined) globals.releaseLimit = orgReleaseLimit;
const orgReleaseAssetLimit = normalizeReleaseAssetLimit(orgOverrides.releaseAssetLimit);
if (orgReleaseAssetLimit !== undefined) globals.releaseAssetLimit = orgReleaseAssetLimit;
const orgReleaseType = normalizeReleaseType(orgOverrides.releaseType);
if (orgReleaseType !== undefined) globals.releaseType = orgReleaseType;
const orgIncludeTags = normalizeReleaseTagPatterns(orgOverrides.releaseIncludeTags);
if (orgIncludeTags !== undefined) globals.releaseIncludeTags = orgIncludeTags;
const orgExcludeTags = normalizeReleaseTagPatterns(orgOverrides.releaseExcludeTags);
if (orgExcludeTags !== undefined) globals.releaseExcludeTags = orgExcludeTags;
return globals;
}, [mirrorOptions, orgOverrides]);
@@ -1201,6 +1209,7 @@ export default function RepositoryTable({
}}
targetKind="repository"
targetName={overridesTarget?.fullName ?? ""}
targetId={overridesTarget?.id ?? null}
value={overridesTarget?.mirrorOverrides ?? null}
inheritedFrom={inheritedMirrorOptions}
inheritedLoading={orgOverridesLoading}
+34
View File
@@ -53,6 +53,11 @@ export const backupStrategyEnum = z.enum([
"block-on-force-push",
]);
// Which releases are mirror candidates: every one, or only the ones the source
// does not flag as a pre-release.
export const releaseTypeEnum = z.enum(["all", "stable"]);
export type ReleaseType = z.infer<typeof releaseTypeEnum>;
/**
* Per-object overrides for the global mirror options in `giteaConfigSchema`.
*
@@ -73,6 +78,22 @@ export const backupStrategyEnum = z.enum([
* assets are uploaded only for the newest N mirrored releases. 0 is a real
* value (notes only); null/absent means inherit.
*
* `releaseType` sits alongside those limits and decides which releases are
* candidates at all (`all`, or `stable` = everything the source does not flag as
* a pre-release). It is a plain optional string rather than a strict enum for
* the same reason as the limits: a stored typo must degrade to "inherit" for
* that field alone. `normalizeReleaseType` does the sanitizing, and the global
* default is `giteaConfigSchema.releaseType`.
*
* `releaseIncludeTags` and `releaseExcludeTags` filter by tag name on top of
* that, for the many projects that never set the pre-release flag but tag
* `v1.2.3-rc.1`. They are `z.unknown()` rather than `z.array(z.string())`
* because a hand-written `"releaseExcludeTags": "*-rc*"` (a bare string, not a
* list) would otherwise fail the parse of the whole object and silently drop
* the LFS opt-out stored next to it. `normalizeReleaseTagPatterns` accepts a
* list of strings and ignores everything else, and an empty list is not a pin:
* clearing the field means inherit.
*
* `skipForks` is organization-tier only: it opts an organization out of
* importing and mirroring forked repositories without touching the global
* `githubConfig.skipForks` switch. Null/absent inherits that global value.
@@ -91,6 +112,9 @@ export const mirrorOverridesSchema = z.object({
mirrorMilestones: z.boolean().nullable().optional(),
releaseLimit: z.number().nullable().optional(),
releaseAssetLimit: z.number().nullable().optional(),
releaseType: z.string().nullable().optional(),
releaseIncludeTags: z.unknown().nullable().optional(),
releaseExcludeTags: z.unknown().nullable().optional(),
skipForks: z.boolean().nullable().optional(),
});
@@ -129,6 +153,15 @@ export const giteaConfigSchema = z.object({
// Upload assets only for the newest N mirrored releases. null/absent means
// every mirrored release gets its assets; 0 means release notes only.
releaseAssetLimit: z.number().nullable().optional(),
// Which releases are candidates: every one, or only the ones not flagged as a
// pre-release. The limit is applied after filtering.
releaseType: releaseTypeEnum.default("all"),
// Tag patterns narrowing that further: include keeps only matching tags,
// exclude drops matching ones. Applied after `releaseType`, before the limit.
// Empty list means no filter, which is what every config stored before this
// option existed behaves.
releaseIncludeTags: z.array(z.string()).default([]),
releaseExcludeTags: z.array(z.string()).default([]),
mirrorMetadata: z.boolean().default(false),
mirrorIssues: z.boolean().default(false),
mirrorPullRequests: z.boolean().default(false),
@@ -341,6 +374,7 @@ export const mirrorJobSchema = z.object({
"synced",
"archived",
"pending-approval",
"warning", // Activity-log alert (e.g. release filters emptied a destination); never a repository status
])
.default("imported"),
message: z.string(),
+32 -1
View File
@@ -11,7 +11,7 @@ import { isSourceProviderKind, type SourceProviderKind } from '@/lib/source-prov
import { DESTINATION_PROVIDER_DEFAULT_URLS, isDestinationProviderKind, type DestinationProviderKind } from '@/lib/destination-kinds';
import { hasDestinationChanged, hasSourceChanged, loadConfigLocks } from '@/lib/config-locks';
import type { SourceUpdateInput } from '@/lib/sources';
import { normalizeReleaseAssetLimit } from '@/lib/utils/mirror-overrides';
import { normalizeReleaseAssetLimit, normalizeReleaseTagPatterns, normalizeReleaseType, splitReleaseTagEntries } from '@/lib/utils/mirror-overrides';
interface EnvConfig {
github: {
@@ -66,6 +66,9 @@ interface EnvConfig {
mirrorMetadata?: boolean;
releaseLimit?: number;
releaseAssetLimit?: number;
releaseType?: string;
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
issueConcurrency?: number;
pullRequestConcurrency?: number;
};
@@ -118,6 +121,15 @@ function parseEnvConfig(): EnvConfig {
const starredLists = process.env.MIRROR_STARRED_LISTS
? process.env.MIRROR_STARRED_LISTS.split(',').map((list) => list.trim()).filter(Boolean)
: undefined;
// Release tag patterns are comma separated, but a comma inside the slash
// form belongs to the regular expression (^v\d{1,3}\.\d+$), so the split
// skips commas inside /.../ - the same rules the dialog applies.
const releaseIncludeTags = process.env.RELEASE_INCLUDE_TAGS
? splitReleaseTagEntries(process.env.RELEASE_INCLUDE_TAGS)
: undefined;
const releaseExcludeTags = process.env.RELEASE_EXCLUDE_TAGS
? splitReleaseTagEntries(process.env.RELEASE_EXCLUDE_TAGS)
: undefined;
return {
github: {
@@ -185,6 +197,9 @@ function parseEnvConfig(): EnvConfig {
mirrorMetadata: process.env.MIRROR_METADATA === 'true',
releaseLimit: process.env.RELEASE_LIMIT ? parseInt(process.env.RELEASE_LIMIT, 10) : undefined,
releaseAssetLimit: process.env.RELEASE_ASSET_LIMIT ? parseInt(process.env.RELEASE_ASSET_LIMIT, 10) : undefined,
releaseType: process.env.RELEASE_TYPE,
releaseIncludeTags,
releaseExcludeTags,
issueConcurrency: process.env.MIRROR_ISSUE_CONCURRENCY ? parseInt(process.env.MIRROR_ISSUE_CONCURRENCY, 10) : undefined,
pullRequestConcurrency: process.env.MIRROR_PULL_REQUEST_CONCURRENCY ? parseInt(process.env.MIRROR_PULL_REQUEST_CONCURRENCY, 10) : undefined,
},
@@ -382,6 +397,22 @@ export async function initializeConfigFromEnv(): Promise<void> {
normalizeReleaseAssetLimit(envConfig.mirror.releaseAssetLimit) ??
existingConfig?.[0]?.giteaConfig?.releaseAssetLimit ??
null,
// all | stable. An unusable value (a typo in the env var)
// falls through to the stored value rather than storing a filter.
releaseType:
normalizeReleaseType(envConfig.mirror.releaseType) ??
existingConfig?.[0]?.giteaConfig?.releaseType ??
'all',
// Comma separated glob patterns (or the /regex/ form). An unset or unusable
// variable keeps what the UI stored, matching how RELEASE_TYPE behaves.
releaseIncludeTags:
normalizeReleaseTagPatterns(envConfig.mirror.releaseIncludeTags) ??
existingConfig?.[0]?.giteaConfig?.releaseIncludeTags ??
[],
releaseExcludeTags:
normalizeReleaseTagPatterns(envConfig.mirror.releaseExcludeTags) ??
existingConfig?.[0]?.giteaConfig?.releaseExcludeTags ??
[],
issueConcurrency: envConfig.mirror.issueConcurrency && envConfig.mirror.issueConcurrency > 0
? envConfig.mirror.issueConcurrency
: existingConfig?.[0]?.giteaConfig?.issueConcurrency ?? 3,
+100 -3
View File
@@ -11,6 +11,7 @@ mock.module("@/lib/helpers", () => ({
}));
const mockMirrorGitHubReleasesToGitea = mock(() => Promise.resolve());
const mockMirrorGiteaSourceReleasesToGitea = mock(() => Promise.resolve());
const mockMirrorGitRepoIssuesToGitea = mock(() => Promise.resolve(undefined));
const mockMirrorGitRepoPullRequestsToGitea = mock(() => Promise.resolve(undefined));
const mockMirrorGitRepoLabelsToGitea = mock(() => Promise.resolve());
@@ -462,6 +463,7 @@ describe("Enhanced Gitea Operations", () => {
mockDb.insert.mockClear();
mockDb.update.mockClear();
mockMirrorGitHubReleasesToGitea.mockClear();
mockMirrorGiteaSourceReleasesToGitea.mockClear();
mockMirrorGitRepoIssuesToGitea.mockClear();
mockMirrorGitRepoPullRequestsToGitea.mockClear();
mockMirrorGitRepoLabelsToGitea.mockClear();
@@ -853,13 +855,14 @@ describe("Enhanced Gitea Operations", () => {
expect(result).toEqual({ success: true });
expect(mockGetGiteaRepoOwnerAsync).toHaveBeenCalled();
expect(mockMirrorGitHubReleasesToGitea).toHaveBeenCalledTimes(1);
const releaseCall = mockMirrorGitHubReleasesToGitea.mock.calls[0][0];
const releaseCall = mockMirrorGitHubReleasesToGitea.mock.calls[0][0] as any;
expect(releaseCall.giteaOwner).toBe("starred");
expect(releaseCall.giteaRepoName).toBe("mirror-repo");
expect(releaseCall.config.githubConfig?.token).toBe("github-token");
expect(releaseCall.octokit).toBeDefined();
// No override anywhere: the resolved default limit is handed through.
expect(releaseCall.releaseLimit).toBe(10);
expect(releaseCall.releaseType).toBe("all");
// Description/topics are reconciled on every sync, against the resolved
// target, with the Gitea repo info already fetched during target
@@ -873,7 +876,7 @@ describe("Enhanced Gitea Operations", () => {
expect(metadataCall.current?.mirror).toBe(true);
});
test("passes the per-repository release limit through to the release mirror", async () => {
test("passes the per-repository release limit and release type through to the release mirror", async () => {
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
@@ -888,6 +891,8 @@ describe("Enhanced Gitea Operations", () => {
defaultOwner: "testuser",
mirrorReleases: true,
releaseLimit: 25,
releaseType: "all",
releaseIncludeTags: ["v*"],
},
};
@@ -902,7 +907,11 @@ describe("Enhanced Gitea Operations", () => {
status: repoStatusEnum.parse("mirrored"),
visibility: "public",
userId: "user123",
mirrorOverrides: { releaseLimit: 3 },
mirrorOverrides: {
releaseLimit: 3,
releaseType: "stable",
releaseExcludeTags: ["*-rc.*"],
},
createdAt: new Date(),
updatedAt: new Date(),
};
@@ -923,6 +932,12 @@ describe("Enhanced Gitea Operations", () => {
expect(mockMirrorGitHubReleasesToGitea).toHaveBeenCalledTimes(1);
const releaseCall = mockMirrorGitHubReleasesToGitea.mock.calls[0][0] as any;
expect(releaseCall.releaseLimit).toBe(3);
// The repository tier wins over the global "all" default.
expect(releaseCall.releaseType).toBe("stable");
// The repository sets no include list, so the global one still applies,
// alongside the repository's own exclude list.
expect(releaseCall.releaseIncludeTags).toEqual(["v*"]);
expect(releaseCall.releaseExcludeTags).toEqual(["*-rc.*"]);
});
test("prefers recorded mirroredLocation when owner resolution changes", async () => {
@@ -1887,6 +1902,88 @@ describe("Enhanced Gitea Operations", () => {
sourceRows.splice(0, sourceRows.length, ...originalRows);
}
});
test("forwards the release filters to a Gitea source release mirror", async () => {
// The Gitea/Forgejo source path has its own release mirror (#440); it has
// to receive the same resolved filters as the GitHub path.
const originalRows = sourceRows.splice(
0,
sourceRows.length,
{
id: "source-gitea",
userId: "user123",
name: "Gitea",
provider: "gitea",
url: "https://git.example.com",
username: "",
token: "source-token",
enabled: true,
createdAt: new Date(0),
updatedAt: new Date(0),
}
);
try {
const config = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: false,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: true,
releaseIncludeTags: ["v*"],
},
} as unknown as Partial<Config>;
const repository = {
id: "repo795",
name: "mirror-repo",
fullName: "group/mirror-repo",
owner: "group",
cloneUrl: "https://git.example.com/group/mirror-repo.git",
isPrivate: false,
isStarred: false,
status: repoStatusEnum.parse("mirrored"),
visibility: "public",
userId: "user123",
sourceId: "source-gitea",
sourceProvider: "gitea",
sourceUrl: "https://git.example.com",
mirrorOverrides: { releaseExcludeTags: ["*-rc.*"] },
createdAt: new Date(),
updatedAt: new Date(),
} as unknown as Repository;
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGiteaSourceReleasesToGitea: mockMirrorGiteaSourceReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
syncRepositoryMetadataToGitea: mockSyncRepositoryMetadataToGitea,
}
);
expect(result).toEqual({ success: true });
expect(mockMirrorGitHubReleasesToGitea).not.toHaveBeenCalled();
expect(mockMirrorGiteaSourceReleasesToGitea).toHaveBeenCalledTimes(1);
const releaseCall = mockMirrorGiteaSourceReleasesToGitea.mock.calls[0][0] as any;
expect(releaseCall.connection.token).toBe("source-token");
expect(releaseCall.releaseIncludeTags).toEqual(["v*"]);
expect(releaseCall.releaseExcludeTags).toEqual(["*-rc.*"]);
} finally {
sourceRows.splice(0, sourceRows.length, ...originalRows);
}
});
});
describe("handleExistingNonMirrorRepo", () => {
+14 -8
View File
@@ -1054,6 +1054,9 @@ export async function syncGiteaRepoEnhanced({
giteaRepoName: repoName,
releaseLimit: mirrorOptions.releaseLimit,
releaseAssetLimit: mirrorOptions.releaseAssetLimit,
releaseType: mirrorOptions.releaseType,
releaseIncludeTags: mirrorOptions.releaseIncludeTags,
releaseExcludeTags: mirrorOptions.releaseExcludeTags,
});
mirrored = true;
}
@@ -1066,14 +1069,17 @@ export async function syncGiteaRepoEnhanced({
repository,
// Carries that instance's own decrypted token, so the release
// listing and the asset downloads authenticate against it.
connection: sourceConnectionFromSource(repoSource, {
userId: config.userId,
}),
giteaOwner: repoOwner,
giteaRepoName: repoName,
releaseLimit: mirrorOptions.releaseLimit,
releaseAssetLimit: mirrorOptions.releaseAssetLimit,
});
connection: sourceConnectionFromSource(repoSource, {
userId: config.userId,
}),
giteaOwner: repoOwner,
giteaRepoName: repoName,
releaseLimit: mirrorOptions.releaseLimit,
releaseAssetLimit: mirrorOptions.releaseAssetLimit,
releaseType: mirrorOptions.releaseType,
releaseIncludeTags: mirrorOptions.releaseIncludeTags,
releaseExcludeTags: mirrorOptions.releaseExcludeTags,
});
mirrored = true;
} else {
console.warn(
+374 -4
View File
@@ -17,7 +17,7 @@
* gitea-org-mirror-destination.test.ts) where the real modules are loaded.
*/
import { afterEach, describe, expect, it, test } from "bun:test";
import { afterEach, describe, expect, it, mock, test } from "bun:test";
import type { Config } from "@/types/config";
import type { Repository } from "@/lib/db/schema";
import type { SourceConnection } from "@/lib/source-providers/types";
@@ -41,6 +41,44 @@ if (!isChild) {
}, 60_000);
}
// Both gitea.ts and helpers.ts bind to the notification service. Replace it in
// the child before importing them so the all-releases-removed warning can be
// captured without loading providers or touching the database.
interface RemovalNotice {
userId: string;
repositoryName: string;
destination: string;
removedCount: number;
}
let removalNotices: RemovalNotice[] = [];
// gitea.ts is the only module that pulls ./helpers into this import graph and
// only uses createMirrorJob from it. Capture the activity rows it writes so the
// all-releases-removed warning can be asserted without touching the database.
interface ActivityJob {
status: string;
message?: string;
details?: string;
skipNotification?: boolean;
skipDuplicateEvent?: boolean;
}
let activityJobs: ActivityJob[] = [];
if (isChild) {
mock.module("@/lib/notification-service", () => ({
triggerAllReleasesRemovedNotification: async (notice: RemovalNotice) => {
removalNotices.push(notice);
},
triggerJobNotification: async () => {},
}));
mock.module("@/lib/helpers", () => ({
createMirrorJob: async (job: ActivityJob) => {
activityJobs.push(job);
return "activity-job";
},
}));
}
const { mirrorGiteaSourceReleasesToGitea, mirrorGitHubReleasesToGitea } = isChild
? await import("@/lib/gitea")
: ({} as typeof import("@/lib/gitea"));
@@ -54,6 +92,8 @@ const realFetch = globalThis.fetch;
afterEach(() => {
globalThis.fetch = realFetch;
removalNotices = [];
activityJobs = [];
});
const config = {
@@ -90,6 +130,8 @@ interface DestinationOptions {
existingRelease?: { id: number; name: string; body: string; tag_name: string };
/** Tags the destination already has. Defaults to every source tag. */
destinationTags?: string[];
/** Releases already on the destination, as the retention pass lists them. */
destinationReleases?: Array<Record<string, unknown>>;
}
function fakeWorld(options: DestinationOptions) {
@@ -161,9 +203,14 @@ function fakeWorld(options: DestinationOptions) {
return json({ id: 77 });
}
// The retention pass removing a release.
if (method === "DELETE" && /\/releases\/\d+$/.test(url)) {
return new Response(null, { status: 204 });
}
// The retention pass lists everything the destination has.
if (method === "GET" && url.startsWith(`${DEST_API}/releases?`)) {
return json([]);
return json(options.destinationReleases ?? []);
}
throw new Error(`unexpected request: ${method} ${url}`);
@@ -173,6 +220,12 @@ function fakeWorld(options: DestinationOptions) {
requests,
creates: () => requests.filter((r) => r.method === "POST" && r.url === `${DEST_API}/releases`),
patches: () => requests.filter((r) => r.method === "PATCH"),
deletes: () => requests.filter((r) => r.method === "DELETE"),
/** Tags of every release the mirror created on the destination. */
createdTags: () =>
requests
.filter((r) => r.method === "POST" && r.url === `${DEST_API}/releases`)
.map((r) => String(JSON.parse(r.body ?? "{}").tag_name)),
uploads: () => requests.filter((r) => r.method === "POST" && r.url.includes("/assets?name=")),
downloads: () =>
requests.filter(
@@ -196,18 +249,39 @@ function giteaRelease(tag: string, extra: Record<string, unknown> = {}) {
};
}
function mirrorFromGitea(connection: SourceConnection) {
function mirrorFromGitea(
connection: SourceConnection,
overrides: {
releaseLimit?: number;
releaseType?: "all" | "stable";
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
} = {}
) {
return mirrorGiteaSourceReleasesToGitea({
config,
repository,
connection,
giteaOwner: DEST_OWNER,
giteaRepoName: DEST_REPO,
releaseLimit: 10,
releaseLimit: overrides.releaseLimit ?? 10,
releaseAssetLimit: null,
releaseType: overrides.releaseType,
releaseIncludeTags: overrides.releaseIncludeTags,
releaseExcludeTags: overrides.releaseExcludeTags,
});
}
/** A destination release as the retention pass reads it. */
function destinationRelease(tag: string, id: number, prerelease: boolean | null) {
return {
id,
tag_name: tag,
created_at: `2026-0${Math.max(1, 9 - id % 9)}-01T00:00:00Z`,
prerelease,
};
}
describe.skipIf(!isChild)("mirrorGiteaSourceReleasesToGitea", () => {
it("creates a missing release with the Gitea/Forgejo header", async () => {
const world = fakeWorld({ sourceReleases: [giteaRelease("v1.0.0")] });
@@ -330,6 +404,302 @@ describe.skipIf(!isChild)("mirrorGiteaSourceReleasesToGitea", () => {
});
});
describe.skipIf(!isChild)("release type filter", () => {
it("mirrors only the releases the source does not flag as pre-release", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v2.1.0-beta.1", { prerelease: true }),
giteaRelease("v2.0.0"),
giteaRelease("v1.0.0"),
],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.createdTags()).toEqual(["v2.0.0", "v1.0.0"]);
});
it("mirrors every release when a stored value names a kind that no longer exists", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v2.1.0-beta.1", { prerelease: true }),
giteaRelease("v2.0.0"),
],
});
// A row saved while the filter offered pre-releases on their own must not
// turn into a filter nobody asked for; it falls back to no type filter.
await mirrorFromGitea(giteaConnection("source-token"), {
releaseType: "prerelease" as unknown as "stable",
});
expect(world.createdTags()).toEqual(["v2.1.0-beta.1", "v2.0.0"]);
});
it("looks further back so the limit counts the releases that match", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v3.0.0-rc1", { prerelease: true }),
giteaRelease("v2.0.0"),
giteaRelease("v1.0.0"),
],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseLimit: 1,
releaseType: "stable",
});
// Asking the source for a single release would leave the pre-release as the
// only candidate; the filter widens the scan instead, so "the newest one"
// is the newest stable one.
expect(world.createdTags()).toEqual(["v2.0.0"]);
const listing = world.requests.find((r) => r.url.startsWith(`${SOURCE_URL}/api/v1/`));
expect(listing?.url).toContain("limit=50&page=1");
});
it("removes destination releases the filter excludes", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0")],
destinationReleases: [
destinationRelease("v1.9.0-beta", 11, true),
destinationRelease("v1.8.0", 12, false),
],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.deletes().map((r) => r.url)).toEqual([`${DEST_API}/releases/11`]);
});
it("keeps a destination release whose pre-release flag is unknown", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0")],
destinationReleases: [destinationRelease("v1.8.0", 12, null)],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.deletes()).toHaveLength(0);
});
it("still runs retention when the filter excludes every source release", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.1.0-beta.1", { prerelease: true })],
destinationReleases: [destinationRelease("v2.1.0-beta.1", 21, true)],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.creates()).toHaveLength(0);
expect(world.deletes().map((r) => r.url)).toEqual([`${DEST_API}/releases/21`]);
});
it("leaves the destination alone when every release is a candidate", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0")],
destinationReleases: [destinationRelease("v1.9.0-beta", 11, true)],
});
await mirrorFromGitea(giteaConnection("source-token"));
expect(world.deletes()).toHaveLength(0);
});
});
describe.skipIf(!isChild)("all releases removed warning", () => {
it("warns when the filters leave the destination with no releases", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.1.0-beta.1", { prerelease: true })],
destinationReleases: [
destinationRelease("v2.0.0-beta", 21, true),
destinationRelease("v1.9.0-beta", 22, true),
],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.deletes()).toHaveLength(2);
expect(removalNotices).toEqual([
{
userId: "user-1",
repositoryName: DEST_REPO,
destination: `${DEST_OWNER}/${DEST_REPO}`,
removedCount: 2,
},
]);
expect(activityJobs).toHaveLength(1);
expect(activityJobs[0]).toMatchObject({
status: "warning",
message: `All releases removed from ${DEST_OWNER}/${DEST_REPO}`,
skipNotification: true,
skipDuplicateEvent: true,
});
});
it("stays quiet while at least one release survives the filters", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0")],
destinationReleases: [
destinationRelease("v2.0.0-beta", 21, true),
destinationRelease("v1.9.0", 22, false),
],
});
await mirrorFromGitea(giteaConnection("source-token"), { releaseType: "stable" });
expect(world.deletes().map((r) => r.url)).toEqual([`${DEST_API}/releases/21`]);
expect(removalNotices).toHaveLength(0);
expect(activityJobs).toHaveLength(0);
});
});
describe.skipIf(!isChild)("release tag pattern filter", () => {
it("skips the source releases an exclude pattern names", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v2.1.0-rc.1"),
giteaRelease("v2.0.0"),
giteaRelease("v1.0.0"),
],
});
// The upstream never flagged anything as a pre-release, which is exactly the
// case the tag filter exists for.
await mirrorFromGitea(giteaConnection("source-token"), {
releaseExcludeTags: ["*-rc.*"],
});
expect(world.createdTags()).toEqual(["v2.0.0", "v1.0.0"]);
});
it("mirrors only the tags an include pattern names", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0"), giteaRelease("v1.2.0"), giteaRelease("v1.0.0")],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseIncludeTags: ["v1.*"],
});
expect(world.createdTags()).toEqual(["v1.2.0", "v1.0.0"]);
});
it("lets exclude win when both lists name a tag", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0"), giteaRelease("v2.0.0-rc.1")],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseIncludeTags: ["v*"],
releaseExcludeTags: ["*-rc.*"],
});
expect(world.createdTags()).toEqual(["v2.0.0"]);
});
it("matches case-insensitively", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("V2.0.0"), giteaRelease("v1.0.0")],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseIncludeTags: ["v2*"],
});
expect(world.createdTags()).toEqual(["V2.0.0"]);
});
it("treats a pattern in slashes as a regular expression", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v1.2.3-rc.1"),
giteaRelease("v1.2.3"),
giteaRelease("nightly-20260101"),
],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseIncludeTags: ["/^v\\d+\\.\\d+\\.\\d+$/"],
});
expect(world.createdTags()).toEqual(["v1.2.3"]);
});
it("widens the scan window for a tag filter alone", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("nightly-20260201"),
giteaRelease("nightly-20260115"),
giteaRelease("v2.0.0"),
],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseLimit: 1,
releaseIncludeTags: ["v*"],
});
// A one-release scan would have found only nightlies and mirrored nothing;
// the filter looks further back so "the newest one" is the newest match.
expect(world.createdTags()).toEqual(["v2.0.0"]);
const listing = world.requests.find((r) => r.url.startsWith(`${SOURCE_URL}/api/v1/`));
expect(listing?.url).toContain("limit=50&page=1");
});
it("removes destination releases whose tag no longer matches", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0")],
destinationReleases: [
// No pre-release flag: a tag pattern needs only the tag name, so unlike
// the type filter it can still clean these up.
destinationRelease("v1.9.0-beta", 11, null),
destinationRelease("v1.8.0", 12, null),
],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseExcludeTags: ["*-beta"],
});
expect(world.deletes().map((r) => r.url)).toEqual([`${DEST_API}/releases/11`]);
});
it("combines with the type filter and the count limit", async () => {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v3.0.0-rc.1", { prerelease: true }),
giteaRelease("v2.9.0-beta.2"),
giteaRelease("v2.0.0"),
giteaRelease("v1.0.0"),
],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseLimit: 1,
releaseType: "stable",
releaseExcludeTags: ["*-beta*"],
});
expect(world.createdTags()).toEqual(["v2.0.0"]);
});
it("ignores a pattern that cannot compile instead of failing the run", async () => {
const world = fakeWorld({
sourceReleases: [giteaRelease("v2.0.0"), giteaRelease("v1.0.0")],
});
await mirrorFromGitea(giteaConnection("source-token"), {
releaseExcludeTags: ["/v1(/"],
});
// The broken pattern is dropped, which leaves no filter at all rather than a
// mirror run that throws.
expect(world.createdTags()).toEqual(["v2.0.0", "v1.0.0"]);
expect(world.deletes()).toHaveLength(0);
});
});
describe.skipIf(!isChild)("mirrorGitHubReleasesToGitea keeps its own header and credentials", () => {
function githubOctokit(releases: Array<Record<string, unknown>>) {
return {
+225 -27
View File
@@ -7,7 +7,7 @@ import {
import { membershipRoleEnum } from "@/types/organizations";
import { Octokit } from "@octokit/rest";
import type { Config } from "@/types/config";
import type { Organization, Repository } from "./db/schema";
import type { Organization, ReleaseType, Repository } from "./db/schema";
import { httpPost, httpGet, httpDelete, httpPut, httpPatch } from "./http-client";
import { createMirrorJob } from "./helpers";
import { db, organizations, repositories } from "./db";
@@ -31,6 +31,7 @@ import {
listSources,
sourceAccessToken,
} from "./sources";
import { triggerAllReleasesRemovedNotification } from "./notification-service";
import { isRateLimitError } from "./rate-limit-gate";
import { assertMirrorCredentialUpdateSupported } from "./gitea-mirror-credentials";
import { persistMetadataPassProgress } from "./metadata-progress-store";
@@ -45,9 +46,16 @@ import {
type MetadataSyncCursor,
} from "./metadata-state";
import {
compileReleaseTagPattern,
compileReleaseTagPatterns,
normalizeReleaseAssetLimit,
normalizeReleaseLimit,
normalizeReleaseTagPatterns,
normalizeReleaseType,
releaseGetsAssets,
releaseMatchesTagFilters,
releaseMatchesType,
releaseScanLimit,
resolveMirrorOptionsForRepository,
resolveOrganizationSkipForks,
type ResolvedMirrorOptions,
@@ -79,6 +87,9 @@ export async function mirrorRepositoryReleases({
giteaRepoName,
releaseLimit,
releaseAssetLimit,
releaseType,
releaseIncludeTags,
releaseExcludeTags,
}: {
config: Partial<Config>;
repository: Repository;
@@ -91,6 +102,9 @@ export async function mirrorRepositoryReleases({
giteaRepoName: string;
releaseLimit?: number;
releaseAssetLimit?: number | null;
releaseType?: ReleaseType;
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
}): Promise<boolean> {
if (sourceConnection.provider === "github") {
if (!octokit) {
@@ -108,6 +122,9 @@ export async function mirrorRepositoryReleases({
giteaRepoName,
releaseLimit,
releaseAssetLimit,
releaseType,
releaseIncludeTags,
releaseExcludeTags,
});
return true;
}
@@ -121,6 +138,9 @@ export async function mirrorRepositoryReleases({
giteaRepoName,
releaseLimit,
releaseAssetLimit,
releaseType,
releaseIncludeTags,
releaseExcludeTags,
});
return true;
}
@@ -1168,6 +1188,9 @@ export const mirrorGithubRepoToGitea = async ({
giteaRepoName: targetRepoName,
releaseLimit: mirrorOptions.releaseLimit,
releaseAssetLimit: mirrorOptions.releaseAssetLimit,
releaseType: mirrorOptions.releaseType,
releaseIncludeTags: mirrorOptions.releaseIncludeTags,
releaseExcludeTags: mirrorOptions.releaseExcludeTags,
});
if (mirrored) {
metadataState.components.releases = true;
@@ -1990,6 +2013,9 @@ export async function mirrorGitHubRepoToGiteaOrg({
giteaRepoName: targetRepoName,
releaseLimit: mirrorOptions.releaseLimit,
releaseAssetLimit: mirrorOptions.releaseAssetLimit,
releaseType: mirrorOptions.releaseType,
releaseIncludeTags: mirrorOptions.releaseIncludeTags,
releaseExcludeTags: mirrorOptions.releaseExcludeTags,
});
if (mirrored) {
metadataState.components.releases = true;
@@ -3822,6 +3848,20 @@ type MirrorReleasesParams = {
* none, `undefined` means resolve it here like the release limit.
*/
releaseAssetLimit?: number | null;
/**
* Already-resolved release type: `all` mirrors every release, `stable` only
* the ones the source does not flag as a pre-release. `undefined` means
* resolve it here like the other two.
*/
releaseType?: ReleaseType;
/**
* Already-resolved tag patterns narrowing the release type filter further:
* `releaseIncludeTags` keeps only tags matching one of them,
* `releaseExcludeTags` drops the ones that do. An empty list means no filter.
* `undefined` means resolve them here like the other three.
*/
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
};
/** The same parameters with the source adapter in place of the GitHub client. */
@@ -3829,6 +3869,14 @@ type MirrorSourceReleasesParams = Omit<MirrorReleasesParams, "octokit"> & {
source: ReleaseSourceAdapter;
};
/**
* The stored tag patterns that can actually be compiled. Used only for the log
* line, so a broken pattern is never named as if it were in effect.
*/
function usableTagPatterns(patterns: readonly string[]): string[] {
return patterns.filter((pattern) => compileReleaseTagPattern(pattern) !== null);
}
/**
* Lock key for one destination repository's releases: the destination base URL
* plus the owner and repository name it is mirrored as, lowercased.
@@ -3988,6 +4036,9 @@ async function mirrorSourceReleasesToGiteaLocked({
giteaRepoName,
releaseLimit: releaseLimitOverride,
releaseAssetLimit: releaseAssetLimitOverride,
releaseType: releaseTypeOverride,
releaseIncludeTags: releaseIncludeTagsOverride,
releaseExcludeTags: releaseExcludeTagsOverride,
}: MirrorSourceReleasesParams) {
if (
!config.giteaConfig?.defaultOwner ||
@@ -4046,14 +4097,66 @@ async function mirrorSourceReleasesToGiteaLocked({
? (normalizeReleaseAssetLimit(releaseAssetLimitOverride) ?? null)
: (await resolveOptions()).releaseAssetLimit;
// Which releases are candidates at all: every one, or only the ones the source
// does not flag as a pre-release.
const releaseType =
normalizeReleaseType(releaseTypeOverride) ??
(await resolveOptions()).releaseType;
// Tag patterns narrow that further, for the projects that never set the
// pre-release flag but tag `v1.2.3-rc.1`. Include keeps only matching tags,
// exclude drops matching ones, and exclude wins when both name a tag.
const releaseIncludeTags =
normalizeReleaseTagPatterns(releaseIncludeTagsOverride) ??
(await resolveOptions()).releaseIncludeTags;
const releaseExcludeTags =
normalizeReleaseTagPatterns(releaseExcludeTagsOverride) ??
(await resolveOptions()).releaseExcludeTags;
const includeTagPatterns = compileReleaseTagPatterns(releaseIncludeTags);
const excludeTagPatterns = compileReleaseTagPatterns(releaseExcludeTags);
const hasTypeFilter = releaseType !== "all";
const hasTagFilter =
includeTagPatterns.length > 0 || excludeTagPatterns.length > 0;
const hasReleaseFilter = hasTypeFilter || hasTagFilter;
// The source lists its own newest releases; everything below this point is
// the same reconciliation whichever host they came from.
const limitedReleases = (
await source.listReleases(sourceOwner, sourceRepo, releaseLimit)
).slice(0, releaseLimit);
// the same reconciliation whichever host they came from. With a filter
// active the mirror looks further back first, so "the newest 10" means the
// newest 10 matching releases rather than whatever survives of the newest 10.
const scanLimit = releaseScanLimit(releaseLimit, releaseType, hasTagFilter);
const fetchedReleases = await source.listReleases(sourceOwner, sourceRepo, scanLimit);
const matchingReleases = fetchedReleases.filter(
(release) =>
releaseMatchesType(release, releaseType) &&
releaseMatchesTagFilters(
release.tag_name,
includeTagPatterns,
excludeTagPatterns
)
);
const limitedReleases = matchingReleases.slice(0, releaseLimit);
const filteredOutCount = fetchedReleases.length - matchingReleases.length;
// Only the patterns that actually compiled are named in the log, so a broken
// one cannot look like it is doing something.
const filterDescriptions: string[] = [];
if (hasTypeFilter) filterDescriptions.push(`only ${releaseType} releases`);
const usableIncludeTags = usableTagPatterns(releaseIncludeTags);
const usableExcludeTags = usableTagPatterns(releaseExcludeTags);
if (usableIncludeTags.length > 0) {
filterDescriptions.push(`only tags matching ${usableIncludeTags.join(", ")}`);
}
if (usableExcludeTags.length > 0) {
filterDescriptions.push(`skipping tags matching ${usableExcludeTags.join(", ")}`);
}
console.log(
`[Releases] Found ${limitedReleases.length} releases (limited to latest ${releaseLimit}; ${
filterDescriptions.length > 0
? `${filterDescriptions.join(", ")}; ${filteredOutCount} filtered out`
: "any release type"
}; ${
releaseAssetLimit === null
? "assets for every release"
: `assets for the newest ${releaseAssetLimit}`
@@ -4062,7 +4165,10 @@ async function mirrorSourceReleasesToGiteaLocked({
if (limitedReleases.length === 0) {
console.log(`[Releases] No releases to mirror for ${repository.fullName}`);
return;
// With a filter active the destination may still hold releases the filter
// excludes, and the retention pass below is what removes them, so only the
// unfiltered case keeps its historical early exit.
if (!hasReleaseFilter) return;
}
let mirroredCount = 0;
@@ -4265,10 +4371,16 @@ async function mirrorSourceReleasesToGiteaLocked({
);
}
// Enforce release retention limit by removing the oldest excess releases from Gitea
// Enforce release retention: first drop the releases the type filter excludes,
// then the oldest releases past the limit.
try {
// Paginate to fetch ALL Gitea releases (API max is 100 per page)
const allGiteaReleases: Array<{ id: number; tag_name: string; created_at: string }> = [];
const allGiteaReleases: Array<{
id: number;
tag_name: string;
created_at: string;
prerelease?: boolean | null;
}> = [];
let cleanupPage = 1;
while (true) {
const pageResponse = await httpGet(
@@ -4290,36 +4402,122 @@ async function mirrorSourceReleasesToGiteaLocked({
cleanupPage++;
}
if (allGiteaReleases.length > releaseLimit) {
const excessCount = allGiteaReleases.length - releaseLimit;
const giteaBaseUrl = config.giteaConfig.url;
const giteaToken = decryptedConfig.giteaConfig.token;
const deleteGiteaRelease = async (
release: { id: number; tag_name: string },
reason: string
) => {
try {
await httpDelete(
`${giteaBaseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${release.id}`,
{
Authorization: `token ${giteaToken}`,
}
);
console.log(`[Releases] Deleted ${reason} release: ${release.tag_name}`);
} catch (deleteError) {
console.error(
`[Releases] Failed to delete ${reason} release ${release.tag_name}: ${
deleteError instanceof Error ? deleteError.message : String(deleteError)
}`
);
}
};
// Releases the filters exclude are no longer mirror candidates, so they leave
// the destination the same way releases past the limit do. A release the
// destination did not report a pre-release flag for is kept: guessing which
// kind it was risks deleting the wrong one. Tag patterns need no such caveat -
// they are judged on the tag name, which every release has.
let retainedReleases = allGiteaReleases;
if (hasReleaseFilter) {
const excluded = allGiteaReleases.filter(
(rel) =>
!releaseMatchesTagFilters(
rel.tag_name ?? "",
includeTagPatterns,
excludeTagPatterns
) ||
(hasTypeFilter &&
typeof rel.prerelease === "boolean" &&
!releaseMatchesType(rel, releaseType))
);
if (excluded.length > 0) {
console.log(
`[Releases] Enforcing release filters (${filterDescriptions.join(", ")}): removing ${excluded.length} release(s) in Gitea that do not match`
);
for (const rel of excluded) {
await deleteGiteaRelease(rel, "excluded");
}
const excludedIds = new Set(excluded.map((rel) => rel.id));
retainedReleases = allGiteaReleases.filter(
(rel) => !excludedIds.has(rel.id)
);
}
}
// The filters emptied a destination that had releases. The sync itself
// succeeded, so no job-failure notification would fire; without this the
// user would only notice the gone releases on the Gitea site. The activity
// log gets its own row (like the force-push alerts) and the push channels
// get the same warning through the "Sync errors" switch.
if (allGiteaReleases.length > 0 && retainedReleases.length === 0) {
const userId = config.userId ?? repository.userId;
const removedCount = allGiteaReleases.length;
void triggerAllReleasesRemovedNotification({
userId,
repositoryName: repository.name,
destination: `${repoOwner}/${repoName}`,
removedCount,
});
// skipNotification keeps the row out of the push notification system:
// triggerAllReleasesRemovedNotification covers that side with its own
// message. skipDuplicateEvent stops the row's `repo-<id>-warning` key
// from swallowing the sync's own completion event on the event stream;
// the terminal update already refreshes the activity view. The row uses
// the activity-only "warning" status so it stands out from plain syncs.
await createMirrorJob({
userId,
repositoryId: repository.id,
repositoryName: repository.name,
status: "warning",
skipNotification: true,
skipDuplicateEvent: true,
message: `All releases removed from ${repoOwner}/${repoName}`,
details:
`${removedCount} release${removedCount === 1 ? "" : "s"} deleted by the release filters; the destination now has none.` +
" Check the include/exclude tag patterns and the pre-release setting.",
}).catch((activityError) => {
console.error(
`[Releases] Failed to record the all-releases-removed activity for ${repository.fullName}: ${
activityError instanceof Error ? activityError.message : String(activityError)
}`
);
});
}
if (retainedReleases.length > releaseLimit) {
const excessCount = retainedReleases.length - releaseLimit;
// Sort by created_at ascending (oldest first) so we delete the oldest excess
const sorted = [...allGiteaReleases].sort(
const sorted = [...retainedReleases].sort(
(a, b) => new Date(a.created_at).getTime() - new Date(b.created_at).getTime()
);
const toDelete = sorted.slice(0, excessCount);
console.log(
`[Releases] Enforcing retention limit (${releaseLimit}): ${allGiteaReleases.length} releases found, removing ${toDelete.length} oldest excess release(s)`
`[Releases] Enforcing retention limit (${releaseLimit}): ${retainedReleases.length} releases found, removing ${toDelete.length} oldest excess release(s)`
);
for (const excess of toDelete) {
try {
await httpDelete(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${excess.id}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
);
console.log(`[Releases] Deleted excess release: ${excess.tag_name}`);
} catch (deleteError) {
console.error(
`[Releases] Failed to delete excess release ${excess.tag_name}: ${
deleteError instanceof Error ? deleteError.message : String(deleteError)
}`
);
}
await deleteGiteaRelease(excess, "excess");
}
}
} catch (cleanupError) {
+2 -2
View File
@@ -1,4 +1,4 @@
import type { RepoStatus } from "@/types/Repository";
import type { ActivityStatus, RepoStatus } from "@/types/Repository";
import { db, mirrorJobs } from "./db";
import { eq, and } from "drizzle-orm";
import { buildInterruptedJobsCondition } from "./interrupted-job-detection";
@@ -31,7 +31,7 @@ export async function createMirrorJob({
repositoryName?: string;
details?: string;
message: string;
status: RepoStatus;
status: ActivityStatus;
jobType?: "mirror" | "sync" | "retry";
batchId?: string;
totalItems?: number;
+52 -1
View File
@@ -18,7 +18,7 @@ mock.module("@/lib/utils/encryption", () => ({
}));
// Import after mocks are set up — db is already mocked via setup.bun.ts
import { sendNotification, testNotification, buildNewRepositoriesNotificationEvent, isNewRepositoryGroupEnabled, MAX_LISTED_NEW_REPOSITORIES, STARRED_GROUP_LABEL } from "./notification-service";
import { sendNotification, testNotification, buildNewRepositoriesNotificationEvent, buildAllReleasesRemovedNotificationEvent, triggerAllReleasesRemovedNotification, isNewRepositoryGroupEnabled, MAX_LISTED_NEW_REPOSITORIES, STARRED_GROUP_LABEL } from "./notification-service";
import type { NotificationConfig } from "@/types/config";
describe("sendNotification", () => {
@@ -508,3 +508,54 @@ describe("isNewRepositoryGroupEnabled", () => {
expect(isNewRepositoryGroupEnabled(legacyRow, "starred")).toBe(true);
});
});
describe("buildAllReleasesRemovedNotificationEvent", () => {
test("names the repository and the destination that lost its releases", () => {
const event = buildAllReleasesRemovedNotificationEvent({
repositoryName: "demo",
destination: "mirrors/demo",
removedCount: 3,
});
expect(event.title).toBe("All Releases Removed: demo");
expect(event.type).toBe("sync_error");
expect(event.message).toContain("3 releases deleted from mirrors/demo");
expect(event.message).toContain("include/exclude tag patterns");
});
test("uses the singular when a single release was removed", () => {
const event = buildAllReleasesRemovedNotificationEvent({
repositoryName: "demo",
destination: "mirrors/demo",
removedCount: 1,
});
expect(event.message).toContain("1 release deleted");
});
});
describe("triggerAllReleasesRemovedNotification", () => {
test("stays silent when nothing was removed", async () => {
await triggerAllReleasesRemovedNotification({
userId: "user-1",
repositoryName: "demo",
destination: "mirrors/demo",
removedCount: 0,
});
expect(mockFetch).toHaveBeenCalledTimes(0);
});
test("never throws when the user has no notification config", async () => {
// The setup db mock resolves to no rows, so the config lookup finds none
// and no provider is contacted.
await triggerAllReleasesRemovedNotification({
userId: "user-1",
repositoryName: "demo",
destination: "mirrors/demo",
removedCount: 2,
});
expect(mockFetch).toHaveBeenCalledTimes(0);
});
});
+59
View File
@@ -370,3 +370,62 @@ export async function triggerNewRepositoriesNotification({
console.error("[NotificationService] New repository notification failed:", error);
}
}
/**
* The "every mirrored release was removed" event.
*
* A tag or type filter change makes the retention pass delete the destination
* releases that no longer match. When nothing survives, the sync itself
* succeeded but the destination lost every release it had, so nothing else
* would alert the user.
*/
export function buildAllReleasesRemovedNotificationEvent({
repositoryName,
destination,
removedCount,
}: {
repositoryName: string;
destination: string;
removedCount: number;
}): NotificationEvent {
const plural = removedCount === 1 ? "" : "s";
return {
title: `All Releases Removed: ${repositoryName}`,
message: [
`${removedCount} release${plural} deleted from ${destination} by the release filters; the destination now has none.`,
" Check the include/exclude tag patterns and the pre-release setting before the next sync.",
].join(""),
type: "sync_error",
};
}
/**
* Warns that the release filters emptied a destination that previously had
* releases. The existing "Sync errors" switch gates the alert.
*
* Called fire-and-forget from the retention pass, so it NEVER throws.
*/
export async function triggerAllReleasesRemovedNotification(args: {
userId: string;
repositoryName: string;
destination: string;
removedCount: number;
}): Promise<void> {
try {
if (args.removedCount <= 0) {
return;
}
const notificationConfig = await loadNotificationConfig(args.userId);
if (!notificationConfig?.enabled || !notificationConfig.notifyOnSyncError) {
return;
}
const event = buildAllReleasesRemovedNotificationEvent(args);
await sendNotification(decryptNotificationConfig(notificationConfig), event);
} catch (error) {
console.error("[NotificationService] All-releases-removed notification failed:", error);
}
}
+11 -1
View File
@@ -17,7 +17,7 @@ import { normalizeDestinationProviderKind } from "@/lib/destination-kinds";
import { z } from "zod";
import { githubConfigSchema, giteaConfigSchema, scheduleConfigSchema, cleanupConfigSchema } from "@/lib/db/schema";
import { parseInterval } from "@/lib/utils/duration-parser";
import { normalizeReleaseAssetLimit } from "@/lib/utils/mirror-overrides";
import { normalizeReleaseAssetLimit, normalizeReleaseTagPatterns, normalizeReleaseType, DEFAULT_RELEASE_TYPE } from "@/lib/utils/mirror-overrides";
import { buildClockCronExpression, normalizeTimezone, parseClockCronExpression } from "@/lib/utils/schedule-utils";
// Use the actual database schema types
@@ -171,6 +171,13 @@ export function mapUiToDbConfig(
// null is a real value here ("assets for every mirrored release"), so
// it is stored as such rather than dropped.
releaseAssetLimit: normalizeReleaseAssetLimit(mirrorOptions.releaseAssetLimit) ?? null,
// An unusable value falls back to "all" rather than storing a filter that
// would hide releases from the user.
releaseType: normalizeReleaseType(mirrorOptions.releaseType) ?? DEFAULT_RELEASE_TYPE,
// Junk entries are dropped one by one rather than rejected wholesale, so a
// stray comma in the input cannot discard the rest of the list.
releaseIncludeTags: normalizeReleaseTagPatterns(mirrorOptions.releaseIncludeTags) ?? [],
releaseExcludeTags: normalizeReleaseTagPatterns(mirrorOptions.releaseExcludeTags) ?? [],
mirrorMetadata: mirrorOptions.mirrorMetadata,
mirrorIssues: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.issues,
mirrorPullRequests: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.pullRequests,
@@ -244,6 +251,9 @@ export function mapDbToUiConfig(dbConfig: any): {
mirrorReleases: dbConfig.giteaConfig?.mirrorReleases || false,
releaseLimit: dbConfig.giteaConfig?.releaseLimit || 10,
releaseAssetLimit: normalizeReleaseAssetLimit(dbConfig.giteaConfig?.releaseAssetLimit) ?? null,
releaseType: normalizeReleaseType(dbConfig.giteaConfig?.releaseType) ?? DEFAULT_RELEASE_TYPE,
releaseIncludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseIncludeTags) ?? [],
releaseExcludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseExcludeTags) ?? [],
mirrorLFS: dbConfig.giteaConfig?.lfs || false,
mirrorMetadata: dbConfig.giteaConfig?.mirrorMetadata || false,
metadataComponents: {
+548 -1
View File
@@ -1,22 +1,37 @@
import { describe, expect, test } from "bun:test";
import {
compileReleaseTagPattern,
compileReleaseTagPatterns,
DEFAULT_RELEASE_LIMIT,
DEFAULT_RELEASE_TYPE,
getMirrorOverrideGating,
hasMirrorOverrides,
listOverriddenKeys,
MAX_RELEASE_TAG_PATTERNS,
MAX_RELEASE_TAG_PATTERN_LENGTH,
MIRROR_GATING_REASONS,
MIRROR_OVERRIDE_CHOICE_KEYS,
MIRROR_OVERRIDE_KEYS,
MIRROR_OVERRIDE_LABELS,
MIRROR_OVERRIDE_LIMIT_KEYS,
MIRROR_OVERRIDE_PATTERN_KEYS,
mirrorOptionsToFlags,
normalizeMirrorOverrides,
normalizeReleaseAssetLimit,
normalizeReleaseLimit,
normalizeReleaseTagPatterns,
normalizeReleaseType,
orgForkSkipDecision,
parseMirrorOverrides,
releaseGetsAssets,
releaseMatchesTagFilters,
releaseMatchesType,
releaseScanLimit,
releaseTagMatchesAny,
RELEASE_SCAN_WINDOW,
resolveMirrorOptions,
resolveOrganizationSkipForks,
splitReleaseTagEntries,
STARRED_CLAMPED_KEYS,
UI_MIRROR_OVERRIDE_KEYS,
} from "./mirror-overrides";
@@ -560,7 +575,7 @@ describe("mirrorOptionsToFlags - client/server shape contract", () => {
mapDbToUiConfig(dbConfig).mirrorOptions
);
const stored = dbConfig.giteaConfig as Record<string, boolean>;
const stored = dbConfig.giteaConfig as unknown as Record<string, boolean>;
expect(flags.lfs).toBe(stored.lfs);
expect(flags.wiki).toBe(stored.wiki);
expect(flags.mirrorIssues).toBe(stored.mirrorIssues);
@@ -1223,3 +1238,535 @@ describe("organization skip forks policy", () => {
expect((resolved as any).skipForks).toBeUndefined();
});
});
describe("release type filter", () => {
describe("normalizeReleaseType", () => {
test("accepts the two known values", () => {
expect(normalizeReleaseType("all")).toBe("all");
expect(normalizeReleaseType("stable")).toBe("stable");
});
test("folds case and padding so a hand-written env var works", () => {
expect(normalizeReleaseType("Stable")).toBe("stable");
expect(normalizeReleaseType(" STABLE ")).toBe("stable");
});
test("rejects unknown words and non-strings", () => {
expect(normalizeReleaseType("main")).toBeUndefined();
expect(normalizeReleaseType("beta")).toBeUndefined();
// The kind filter never offered pre-releases on their own, but a value
// saved while it did must fall through to the next tier rather than pin
// something the mirror no longer does.
expect(normalizeReleaseType("prerelease")).toBeUndefined();
expect(normalizeReleaseType("")).toBeUndefined();
expect(normalizeReleaseType(true)).toBeUndefined();
expect(normalizeReleaseType(1)).toBeUndefined();
expect(normalizeReleaseType(null)).toBeUndefined();
expect(normalizeReleaseType(undefined)).toBeUndefined();
});
});
describe("releaseMatchesType", () => {
test("all accepts every release", () => {
expect(releaseMatchesType({ prerelease: true }, "all")).toBe(true);
expect(releaseMatchesType({ prerelease: false }, "all")).toBe(true);
expect(releaseMatchesType({}, "all")).toBe(true);
});
test("stable keeps everything the source does not flag", () => {
expect(releaseMatchesType({ prerelease: false }, "stable")).toBe(true);
expect(releaseMatchesType({ prerelease: true }, "stable")).toBe(false);
});
test("a release with no flag counts as a normal release", () => {
// Dropping everything because a source omits the flag would silently
// empty a mirror, so unknown is treated as "not a pre-release".
expect(releaseMatchesType({}, "stable")).toBe(true);
expect(releaseMatchesType({ prerelease: null }, "stable")).toBe(true);
});
test("a filter splits a newest-first list", () => {
const releases = [
{ prerelease: true },
{ prerelease: false },
{ prerelease: true },
{},
];
expect(releases.filter((r) => releaseMatchesType(r, "stable"))).toHaveLength(2);
expect(releases.filter((r) => releaseMatchesType(r, "all"))).toHaveLength(4);
});
});
describe("releaseScanLimit", () => {
test("without a filter the limit is the window", () => {
expect(releaseScanLimit(10, "all")).toBe(10);
});
test("a filter looks back at least the scan window", () => {
expect(releaseScanLimit(3, "stable")).toBe(RELEASE_SCAN_WINDOW);
expect(releaseScanLimit(1, "stable")).toBe(RELEASE_SCAN_WINDOW);
});
test("a tag filter widens the window the same way", () => {
expect(releaseScanLimit(1, "all", true)).toBe(RELEASE_SCAN_WINDOW);
expect(releaseScanLimit(1, "all", false)).toBe(1);
});
test("a limit bigger than the window still wins", () => {
expect(releaseScanLimit(RELEASE_SCAN_WINDOW + 50, "stable")).toBe(
RELEASE_SCAN_WINDOW + 50
);
});
});
describe("release tag patterns", () => {
test("normalizeReleaseTagPatterns trims, dedupes and drops junk entries", () => {
expect(
normalizeReleaseTagPatterns([" v1.0 ", "", 5, null, "v1.0", "V1.0", "rc"])
).toEqual(["v1.0", "rc"]);
});
test("an empty or unusable list is absent, not an empty pin", () => {
// Clearing the box means inherit, so an empty list has to fall through to
// the next tier rather than pinning "no patterns" here.
expect(normalizeReleaseTagPatterns([])).toBeUndefined();
expect(normalizeReleaseTagPatterns(["", " "])).toBeUndefined();
expect(normalizeReleaseTagPatterns("v*")).toBeUndefined();
expect(normalizeReleaseTagPatterns(null)).toBeUndefined();
expect(normalizeReleaseTagPatterns(undefined)).toBeUndefined();
});
test("over-long patterns are dropped and the list is capped", () => {
expect(
normalizeReleaseTagPatterns(["x".repeat(MAX_RELEASE_TAG_PATTERN_LENGTH + 1)])
).toBeUndefined();
const many = Array.from(
{ length: MAX_RELEASE_TAG_PATTERNS + 5 },
(_, index) => `v${index}`
);
expect(normalizeReleaseTagPatterns(many)).toHaveLength(MAX_RELEASE_TAG_PATTERNS);
});
test("a plain pattern is a whole-tag wildcard, case-insensitive", () => {
expect(compileReleaseTagPattern("*-rc.*")?.test("V1.0-rc.1")).toBe(true);
expect(compileReleaseTagPattern("*-rc.*")?.test("v1.0.0")).toBe(false);
expect(compileReleaseTagPattern("?")?.test("a")).toBe(true);
expect(compileReleaseTagPattern("?")?.test("ab")).toBe(false);
expect(compileReleaseTagPattern("RC")?.test("rc")).toBe(true);
});
test("a wildcard cannot start or end in the middle of a tag", () => {
expect(compileReleaseTagPattern("rc")?.test("v1.0-rc.1")).toBe(false);
expect(compileReleaseTagPattern("*rc*")?.test("v1.0-rc.1")).toBe(true);
});
test("dots and brackets are literal in a glob", () => {
expect(compileReleaseTagPattern("v1.0")?.test("v1.0")).toBe(true);
expect(compileReleaseTagPattern("v1.0")?.test("v1x0")).toBe(false);
expect(compileReleaseTagPattern("[bet]")?.test("[bet]")).toBe(true);
expect(compileReleaseTagPattern("[bet]")?.test("b")).toBe(false);
});
test("the slash form is a regular expression", () => {
const strict = compileReleaseTagPattern("/^v\\d+\\.\\d+\\.\\d+$/");
expect(strict?.test("v1.2.3")).toBe(true);
expect(strict?.test("v1.2.3-rc.1")).toBe(false);
expect(compileReleaseTagPattern("/rc/")?.test("v2.0-rc2")).toBe(true);
});
test("a regex keeps its own flags but always ignores case", () => {
expect(compileReleaseTagPattern("/^V\\d+$/i")?.test("v7")).toBe(true);
expect(compileReleaseTagPattern("/^V\\d+$/")?.test("v7")).toBe(true);
});
test("a pattern that cannot compile is dropped rather than thrown", () => {
// One typo in a stored pattern must not stop a mirror run.
expect(compileReleaseTagPattern("/v1(/")).toBeNull();
expect(compileReleaseTagPattern("")).toBeNull();
expect(compileReleaseTagPattern(true)).toBeNull();
expect(compileReleaseTagPatterns(["v*", "/v1(/", ""])).toHaveLength(1);
});
test("an unterminated slash pattern is rejected, not treated as a glob", () => {
// /... with no closing slash is an unfinished regex. Globbing it would
// store a pattern that silently matches nothing.
expect(compileReleaseTagPattern("/^v\\d{1")).toBeNull();
expect(compileReleaseTagPatterns(["/^v\\d{1"])).toHaveLength(0);
});
describe("splitReleaseTagEntries", () => {
test("splits plain globs on commas and trims away empty entries", () => {
expect(splitReleaseTagEntries(" v* , *-rc.* ,, ")).toEqual(["v*", "*-rc.*"]);
});
test("keeps commas inside the regex form intact", () => {
// The bug this guards: a regex quantifier like {1,3} split into two
// literal globs that match nothing, silently dropping every release.
const entries = splitReleaseTagEntries("/^v\\d{1,3}\\.\\d+$/, v2.*");
expect(entries).toEqual(["/^v\\d{1,3}\\.\\d+$/", "v2.*"]);
expect(compileReleaseTagPattern(entries[0])?.test("v123.4")).toBe(true);
expect(compileReleaseTagPattern(entries[0])?.test("v1234.4")).toBe(false);
expect(compileReleaseTagPattern(entries[1])?.test("v2.0")).toBe(true);
});
test("a backslash keeps an escaped slash from closing the regex early", () => {
expect(splitReleaseTagEntries("/a\\/b,c/, d")).toEqual(["/a\\/b,c/", "d"]);
});
test("an unterminated regex swallows the rest so the compile step flags it", () => {
const entries = splitReleaseTagEntries("/foo, /bar");
expect(entries).toEqual(["/foo, /bar"]);
expect(compileReleaseTagPattern(entries[0])).toBeNull();
});
});
test("include restricts, exclude wins when both name a tag", () => {
const include = compileReleaseTagPatterns(["v*"]);
const exclude = compileReleaseTagPatterns(["*-rc.*"]);
expect(releaseMatchesTagFilters("v1.2.3", include, exclude)).toBe(true);
expect(releaseMatchesTagFilters("next-1.2", include, exclude)).toBe(false);
expect(releaseMatchesTagFilters("v1.0-rc.1", include, exclude)).toBe(false);
});
test("no patterns means every tag passes", () => {
expect(releaseMatchesTagFilters("anything", [], [])).toBe(true);
expect(releaseTagMatchesAny("v1", [])).toBe(false);
});
test("a release with no tag name still evaluates", () => {
expect(releaseMatchesTagFilters("", compileReleaseTagPatterns(["*"]), [])).toBe(true);
expect(releaseMatchesTagFilters("", compileReleaseTagPatterns(["v*"]), [])).toBe(false);
});
});
describe("release tag pattern tiers and overrides", () => {
test("no tier means no filter", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true }),
repository: makeRepo(),
});
expect(resolved.releaseIncludeTags).toEqual([]);
expect(resolved.releaseExcludeTags).toEqual([]);
});
test("organization beats global, repository beats organization", () => {
const config = makeConfig({
mirrorReleases: true,
releaseExcludeTags: ["*-rc.*"],
});
expect(
resolveMirrorOptions({ config, repository: makeRepo() }).releaseExcludeTags
).toEqual(["*-rc.*"]);
expect(
resolveMirrorOptions({
config,
repository: makeRepo(),
orgOverrides: { releaseExcludeTags: ["*-beta.*"] },
}).releaseExcludeTags
).toEqual(["*-beta.*"]);
expect(
resolveMirrorOptions({
config,
repository: makeRepo({
mirrorOverrides: { releaseExcludeTags: ["*-alpha.*"] },
}),
orgOverrides: { releaseExcludeTags: ["*-beta.*"] },
}).releaseExcludeTags
).toEqual(["*-alpha.*"]);
});
test("an empty repository list inherits rather than clearing", () => {
// The documented consequence of "an empty list is not a pin": a repository
// cannot switch off a pattern its organization set.
expect(
resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseExcludeTags: ["*-rc.*"] }),
repository: makeRepo({ mirrorOverrides: { releaseExcludeTags: [] } }),
}).releaseExcludeTags
).toEqual(["*-rc.*"]);
});
test("include and exclude resolve independently of each other", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseIncludeTags: ["v*"] }),
repository: makeRepo({
mirrorOverrides: { releaseExcludeTags: ["*-rc.*"] },
}),
});
expect(resolved.releaseIncludeTags).toEqual(["v*"]);
expect(resolved.releaseExcludeTags).toEqual(["*-rc.*"]);
});
test("junk entries are dropped one at a time, not the whole list", () => {
expect(
resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true }),
repository: makeRepo({
mirrorOverrides: { releaseIncludeTags: ["v*", 7, "", "V*", null] },
}),
}).releaseIncludeTags
).toEqual(["v*"]);
});
test("a pinned pattern list counts as an override", () => {
expect(hasMirrorOverrides({ releaseExcludeTags: ["v*"] })).toBe(true);
expect(listOverriddenKeys({ releaseIncludeTags: ["v*"] })).toEqual([
"releaseIncludeTags",
]);
});
test("empty and junk pattern lists do not count", () => {
expect(hasMirrorOverrides({ releaseIncludeTags: [] })).toBe(false);
expect(hasMirrorOverrides({ releaseExcludeTags: null })).toBe(false);
expect(listOverriddenKeys({ releaseExcludeTags: [7, ""] })).toEqual([]);
});
test("normalizeMirrorOverrides keeps the usable patterns", () => {
expect(
normalizeMirrorOverrides({ releaseIncludeTags: [" v* ", "v*", 5] })
).toEqual({ releaseIncludeTags: ["v*"] });
expect(normalizeMirrorOverrides({ releaseIncludeTags: [] })).toBeNull();
});
test("a junk pattern entry does not take the rest of the overrides with it", () => {
expect(
parseMirrorOverrides({ releaseIncludeTags: ["v*", 7], lfs: false })
).toEqual({ releaseIncludeTags: ["v*", 7], lfs: false });
expect(
normalizeMirrorOverrides({ releaseIncludeTags: ["v*", 7], lfs: false })
).toEqual({ releaseIncludeTags: ["v*"], lfs: false });
});
test("the pattern keys are their own group, with labels", () => {
expect(MIRROR_OVERRIDE_PATTERN_KEYS).toEqual([
"releaseIncludeTags",
"releaseExcludeTags",
]);
expect(MIRROR_OVERRIDE_KEYS).not.toContain("releaseIncludeTags");
expect(MIRROR_OVERRIDE_LIMIT_KEYS).not.toContain("releaseExcludeTags");
expect(MIRROR_OVERRIDE_CHOICE_KEYS).not.toContain("releaseExcludeTags");
for (const key of MIRROR_OVERRIDE_PATTERN_KEYS) {
expect(MIRROR_OVERRIDE_LABELS[key]).toBeTruthy();
}
});
test("mirrorOptionsToFlags exposes the lists for the dialog hints", () => {
expect(mirrorOptionsToFlags({ mirrorReleases: true }).releaseIncludeTags).toEqual([]);
expect(
mirrorOptionsToFlags({ releaseExcludeTags: ["*-rc.*"] }).releaseExcludeTags
).toEqual(["*-rc.*"]);
});
test("the pattern boxes are gated exactly like the release limit", () => {
const releasesOff = getMirrorOverrideGating({
targetKind: "repository",
effective: { mirrorReleases: false },
});
expect(releasesOff.releaseIncludeTags).toBe(MIRROR_GATING_REASONS.releasesOff);
expect(releasesOff.releaseExcludeTags).toBe(MIRROR_GATING_REASONS.releasesOff);
const starred = getMirrorOverrideGating({
targetKind: "repository",
isStarred: true,
starredCodeOnly: true,
effective: { mirrorReleases: true },
});
expect(starred.releaseIncludeTags).toBe(MIRROR_GATING_REASONS.starredCodeOnly);
const pushTarget = getMirrorOverrideGating({
targetKind: "repository",
effective: { mirrorReleases: true },
destinationProvider: "gitlab",
});
expect(pushTarget.releaseIncludeTags).toBe(MIRROR_GATING_REASONS.pushTarget);
expect(pushTarget.releaseExcludeTags).toBe(MIRROR_GATING_REASONS.pushTarget);
const editable = getMirrorOverrideGating({
targetKind: "organization",
effective: { mirrorReleases: true },
});
expect(editable.releaseIncludeTags).toBeUndefined();
expect(editable.releaseExcludeTags).toBeUndefined();
});
});
describe("resolveMirrorOptions", () => {
test("defaults to every release when no tier pins a type", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true }),
repository: makeRepo(),
});
expect(resolved.releaseType).toBe(DEFAULT_RELEASE_TYPE);
expect(resolved.releaseType).toBe("all");
});
test("the global value applies when nothing overrides it", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseType: "stable" }),
repository: makeRepo(),
});
expect(resolved.releaseType).toBe("stable");
});
test("organization beats global, repository beats organization", () => {
const config = makeConfig({ mirrorReleases: true, releaseType: "all" });
expect(
resolveMirrorOptions({
config,
repository: makeRepo(),
orgOverrides: { releaseType: "stable" },
}).releaseType
).toBe("stable");
expect(
resolveMirrorOptions({
config,
repository: makeRepo({ mirrorOverrides: { releaseType: "all" } }),
orgOverrides: { releaseType: "stable" },
}).releaseType
).toBe("all");
});
test("null in an override means inherit", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseType: "stable" }),
repository: makeRepo({ mirrorOverrides: { releaseType: null } }),
orgOverrides: { releaseType: null },
});
expect(resolved.releaseType).toBe("stable");
});
test("an unusable value falls through to the next tier", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseType: "all" }),
repository: makeRepo({ mirrorOverrides: { releaseType: "beta" } }),
orgOverrides: { releaseType: "stable" },
});
expect(resolved.releaseType).toBe("stable");
});
test("a tier with nothing usable lands on the default rather than a filter", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseType: "weekly" }),
repository: makeRepo({ mirrorOverrides: { releaseType: "beta" } }),
});
expect(resolved.releaseType).toBe(DEFAULT_RELEASE_TYPE);
});
test("the release type is independent of the release limit", () => {
const resolved = resolveMirrorOptions({
config: makeConfig({ mirrorReleases: true, releaseLimit: 3 }),
repository: makeRepo({ mirrorOverrides: { releaseType: "stable" } }),
});
expect(resolved.releaseLimit).toBe(3);
expect(resolved.releaseType).toBe("stable");
});
});
describe("override helpers", () => {
test("a pinned release type counts as an override", () => {
expect(hasMirrorOverrides({ releaseType: "stable" })).toBe(true);
expect(listOverriddenKeys({ releaseType: "all" })).toEqual(["releaseType"]);
});
test("null and unusable values do not count", () => {
expect(hasMirrorOverrides({ releaseType: null })).toBe(false);
expect(hasMirrorOverrides({ releaseType: "beta" })).toBe(false);
expect(listOverriddenKeys({ releaseType: "beta" })).toEqual([]);
});
test("normalizeMirrorOverrides folds case and drops junk", () => {
expect(normalizeMirrorOverrides({ releaseType: "Stable" })).toEqual({
releaseType: "stable",
});
expect(normalizeMirrorOverrides({ releaseType: "beta" })).toBeNull();
expect(
normalizeMirrorOverrides({ releaseType: null, lfs: true })
).toEqual({ lfs: true });
});
test("a junk release type does not take the rest of the overrides with it", () => {
// The stored column is loose on purpose: one bad field must not make the
// whole object unparseable and silently drop the LFS opt-out next to it.
expect(parseMirrorOverrides({ releaseType: "beta", lfs: false })).toEqual({
releaseType: "beta",
lfs: false,
});
expect(normalizeMirrorOverrides({ releaseType: "beta", lfs: false })).toEqual({
lfs: false,
});
});
test("the choice keys and labels agree", () => {
expect(MIRROR_OVERRIDE_CHOICE_KEYS).toEqual(["releaseType"]);
for (const key of MIRROR_OVERRIDE_CHOICE_KEYS) {
expect(MIRROR_OVERRIDE_LABELS[key]).toBeTruthy();
}
});
test("the choice keys are not walked as flags or limits", () => {
expect(MIRROR_OVERRIDE_KEYS).not.toContain("releaseType");
expect(MIRROR_OVERRIDE_LIMIT_KEYS).not.toContain("releaseType");
});
});
describe("mirrorOptionsToFlags", () => {
test("defaults to every release when the API sends no type", () => {
expect(mirrorOptionsToFlags({ mirrorReleases: true }).releaseType).toBe("all");
expect(
mirrorOptionsToFlags({ mirrorReleases: true, releaseType: null as any }).releaseType
).toBe("all");
});
test("passes the stored value through", () => {
expect(mirrorOptionsToFlags({ releaseType: "stable" }).releaseType).toBe("stable");
// A value from a release that no longer offers this kind lands on the
// default instead of turning into a filter nothing asked for.
expect(mirrorOptionsToFlags({ releaseType: "prerelease" as any }).releaseType).toBe(
"all"
);
});
});
describe("getMirrorOverrideGating", () => {
test("the release type is disabled with the release limit when releases are off", () => {
const gating = getMirrorOverrideGating({
targetKind: "repository",
effective: { mirrorReleases: false },
});
expect(gating.releaseType).toBe(MIRROR_GATING_REASONS.releasesOff);
expect(gating.releaseLimit).toBe(MIRROR_GATING_REASONS.releasesOff);
});
test("the release type inherits the starred reason", () => {
const gating = getMirrorOverrideGating({
targetKind: "repository",
isStarred: true,
starredCodeOnly: true,
effective: { mirrorReleases: true },
});
expect(gating.releaseType).toBe(MIRROR_GATING_REASONS.starredCodeOnly);
});
test("the release type is editable while releases are on", () => {
const gating = getMirrorOverrideGating({
targetKind: "organization",
effective: { mirrorReleases: true },
});
expect(gating.releaseType).toBeUndefined();
});
test("a push target disables the release type", () => {
const gating = getMirrorOverrideGating({
targetKind: "repository",
effective: { mirrorReleases: true },
destinationProvider: "github",
});
expect(gating.releaseType).toBe(MIRROR_GATING_REASONS.pushTarget);
});
});
});
+384 -9
View File
@@ -1,5 +1,5 @@
import type { Config } from "@/types/config";
import type { MirrorOverrides, Repository } from "@/lib/db/schema";
import type { MirrorOverrides, ReleaseType, Repository } from "@/lib/db/schema";
import { mirrorOverridesSchema } from "@/lib/db/schema";
import {
normalizeSourceProviderKind,
@@ -38,6 +38,36 @@ export const MIRROR_OVERRIDE_LIMIT_KEYS = ["releaseLimit", "releaseAssetLimit"]
export type MirrorOverrideLimitKey = (typeof MIRROR_OVERRIDE_LIMIT_KEYS)[number];
/**
* The choice-based mirror options that can be overridden per organization and
* per repository. Kept apart from the flags and the limits because both the
* tri-state toggles and the numeric rows in the dialog assume those shapes.
*
* `releaseType` decides which releases are candidates at all; `releaseLimit`
* then decides how many of the matching ones are kept.
*/
export const MIRROR_OVERRIDE_CHOICE_KEYS = ["releaseType"] as const;
export type MirrorOverrideChoiceKey =
(typeof MIRROR_OVERRIDE_CHOICE_KEYS)[number];
/**
* The tag-pattern mirror options that can be overridden per organization and
* per repository. Kept apart from the flags, limits and choices because all of
* those assume a scalar value; these hold a list of patterns.
*
* `releaseIncludeTags` keeps only the releases whose tag matches one of the
* patterns, `releaseExcludeTags` drops the ones that do. Both run after
* `releaseType` and before `releaseLimit`.
*/
export const MIRROR_OVERRIDE_PATTERN_KEYS = [
"releaseIncludeTags",
"releaseExcludeTags",
] as const;
export type MirrorOverridePatternKey =
(typeof MIRROR_OVERRIDE_PATTERN_KEYS)[number];
/**
* Organization-tier selection overrides: per-organization switches that steer
* which source repositories are imported and mirrored, not what a mirror
@@ -53,6 +83,27 @@ export type OrgSelectionOverrideKey = (typeof ORG_SELECTION_OVERRIDE_KEYS)[numbe
/** Matches `giteaConfigSchema.releaseLimit`'s default. */
export const DEFAULT_RELEASE_LIMIT = 10;
/** Matches `giteaConfigSchema.releaseType`'s default. */
export const DEFAULT_RELEASE_TYPE: ReleaseType = "all";
/**
* How many releases the mirror looks at when any release filter is active.
*
* Filtering happens before the limit, so without a wider window a repository
* whose newest releases are all filtered out would mirror nothing at all when
* asked for the other kind. 100 is GitHub's maximum page size, so it costs one
* extra page at most.
*/
export const RELEASE_SCAN_WINDOW = 100;
/**
* Bounds on stored tag patterns. They are compiled into regular expressions on
* every mirror run, so a hand-edited row holding thousands of them cannot be
* allowed to turn a sync into a regex benchmark.
*/
export const MAX_RELEASE_TAG_PATTERNS = 20;
export const MAX_RELEASE_TAG_PATTERN_LENGTH = 200;
/**
* Fully resolved mirror options: every flag has a definite boolean value, the
* release limit a definite positive integer, and the asset limit either a
@@ -61,6 +112,9 @@ export const DEFAULT_RELEASE_LIMIT = 10;
export type ResolvedMirrorOptions = Record<MirrorOverrideKey, boolean> & {
releaseLimit: number;
releaseAssetLimit: number | null;
releaseType: ReleaseType;
releaseIncludeTags: string[];
releaseExcludeTags: string[];
};
/**
@@ -71,6 +125,9 @@ export type ResolvedMirrorOptions = Record<MirrorOverrideKey, boolean> & {
export type InheritedMirrorOptions = Partial<Record<MirrorOverrideKey, boolean>> & {
releaseLimit?: number;
releaseAssetLimit?: number | null;
releaseType?: ReleaseType;
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
skipForks?: boolean;
};
@@ -99,6 +156,245 @@ export function normalizeReleaseAssetLimit(value: unknown): number | undefined {
return limit >= 0 ? limit : undefined;
}
const RELEASE_TYPES: readonly ReleaseType[] = ["all", "stable"];
/**
* Sanitize a release type from any tier into one of the known values, or
* `undefined` when it is absent or unusable (so the next tier out is
* consulted). Case is folded so a hand-written `RELEASE_TYPE=Stable` env var
* behaves the same as the value the UI writes.
*/
export function normalizeReleaseType(value: unknown): ReleaseType | undefined {
if (typeof value !== "string") return undefined;
const type = value.trim().toLowerCase();
return RELEASE_TYPES.find((candidate) => candidate === type);
}
/**
* Whether a release belongs to the configured type.
*
* The `prerelease` flag is the only signal: GitHub and Gitea both report it.
* A source that does not report it (`undefined`/`null`) counts as a normal
* release, which keeps `stable` from silently dropping everything.
*/
export function releaseMatchesType(
release: { prerelease?: boolean | null },
releaseType: ReleaseType
): boolean {
if (releaseType === "all") return true;
return release.prerelease !== true;
}
/**
* Sanitize a tag-pattern list from any tier into a usable list, or `undefined`
* when it is absent or unusable (so the next tier out is consulted). Entries
* that are not non-empty strings are ignored one by one, duplicates collapse,
* and the list is capped so a hand-edited row cannot grow past what a mirror
* run should compile.
*
* An empty list is deliberately not a pin: clearing the field in the UI means
* "inherit", the same way an empty release-limit box does.
*/
export function normalizeReleaseTagPatterns(value: unknown): string[] | undefined {
if (!Array.isArray(value)) return undefined;
const seen = new Set<string>();
const patterns: string[] = [];
for (const entry of value) {
if (typeof entry !== "string") continue;
const pattern = entry.trim();
if (!pattern || pattern.length > MAX_RELEASE_TAG_PATTERN_LENGTH) continue;
const key = pattern.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
patterns.push(pattern);
if (patterns.length >= MAX_RELEASE_TAG_PATTERNS) break;
}
return patterns.length > 0 ? patterns : undefined;
}
/**
* Split a tag rules box into entries on commas that sit OUTSIDE a /.../ pair.
*
* Regex bodies legitimately contain commas - `^v\d{1,3}\.\d+$` is full of them -
* so a naive split(",") turns one regex into two globs that match nothing. A
* backslash escapes the next character, which keeps `\/` from closing the body
* early. Slashes that do not open an entry (mid-glob, as in `a/b`) are literal.
*
* An unterminated `/` swallows the rest of the text into one entry; the compile
* step then rejects it, so the dialog can flag the typo instead of storing a
* glob that silently matches nothing.
*/
export function splitReleaseTagEntries(text: string): string[] {
const entries: string[] = [];
let current = "";
let insideRegex = false;
for (let i = 0; i < text.length; i++) {
const char = text[i];
if (insideRegex) {
if (char === "\\") {
current += char;
i++;
if (i < text.length) current += text[i];
continue;
}
if (char === "/") insideRegex = false;
current += char;
continue;
}
if (char === "/" && current.trim() === "") {
insideRegex = true;
current += char;
continue;
}
if (char === ",") {
entries.push(current);
current = "";
continue;
}
current += char;
}
entries.push(current);
return entries.map((entry) => entry.trim()).filter(Boolean);
}
/** Escape everything that has a meaning inside a regular expression. */
function escapeRegexChars(value: string): string {
return value.replace(/[.+^${}()|[\]\\]/g, "\\$&");
}
/** Turn a whole-string glob into a regular expression source. */
function globToRegexSource(glob: string): string {
let source = "";
for (const char of glob) {
if (char === "*") source += ".*";
else if (char === "?") source += ".";
else source += escapeRegexChars(char);
}
return `^(?:${source})$`;
}
/** Keep any flags the user asked for, but always match case-insensitively. */
function releaseTagPatternFlags(flags: string): string {
const cleaned = flags
.split("")
.filter((flag) => "imsuy".includes(flag))
.join("");
return cleaned.includes("i") ? cleaned : `${cleaned}i`;
}
/**
* Compile one stored tag pattern.
*
* A pattern wrapped in slashes (`/v1\.\d+$/`) is a JavaScript regular
* expression, which is the escape hatch for whoever needs it. Anything else is a
* glob matched against the whole tag name: `*` is any run of characters, `?` is
* exactly one, everything else is literal. So `*-rc.*` skips `v1.0-rc.1` and
* `v2.0-rc2`, while plain `rc` matches nothing but a tag spelled `rc`. Case is
* ignored either way, because upstreams spell their tags however they like.
*
* Returns `null` for a pattern that cannot be used, including a regular
* expression that does not compile. Dropping it beats throwing: one typo must
* not stop a mirror run.
*/
export function compileReleaseTagPattern(pattern: unknown): RegExp | null {
if (typeof pattern !== "string") return null;
const trimmed = pattern.trim();
if (!trimmed) return null;
if (trimmed.length > 2 && trimmed.startsWith("/")) {
const closing = trimmed.lastIndexOf("/");
if (closing > 0) {
const trailing = trimmed.slice(closing + 1);
// Anything after the closing slash has to be regex flags. Junk like the
// `bar` in `/foo, /bar` means the "regex" was really an unterminated one
// that swallowed a later slash, so reject it rather than silently
// dropping the junk and compiling something the user never wrote.
if (/^[imsuy]*$/.test(trailing)) {
const body = trimmed.slice(1, closing);
if (body) {
try {
return new RegExp(body, releaseTagPatternFlags(trailing));
} catch {
return null;
}
}
}
}
// A leading slash with no closing slash is an unfinished regex, not a tag
// spelled with a leading slash (tags cannot start with one). Rejecting it
// lets the dialog flag the typo instead of keeping a glob that silently
// matches nothing.
return null;
}
try {
return new RegExp(globToRegexSource(trimmed), "i");
} catch {
return null;
}
}
/** Compile a whole list, dropping the entries that cannot be used. */
export function compileReleaseTagPatterns(
patterns: readonly unknown[] | null | undefined
): RegExp[] {
if (!Array.isArray(patterns)) return [];
const compiled: RegExp[] = [];
for (const pattern of patterns) {
const regex = compileReleaseTagPattern(pattern);
if (regex) compiled.push(regex);
}
return compiled;
}
/** True when at least one compiled pattern matches the tag. */
export function releaseTagMatchesAny(
tag: string,
patterns: readonly RegExp[]
): boolean {
return patterns.some((pattern) => pattern.test(tag ?? ""));
}
/**
* Whether a tag survives the include and exclude lists.
*
* Exclude wins when a tag is named by both, because "only these, except that
* one" is the only reading that makes such a configuration mean anything.
*/
export function releaseMatchesTagFilters(
tag: string,
includePatterns: readonly RegExp[],
excludePatterns: readonly RegExp[]
): boolean {
if (includePatterns.length > 0 && !releaseTagMatchesAny(tag, includePatterns)) {
return false;
}
if (excludePatterns.length > 0 && releaseTagMatchesAny(tag, excludePatterns)) {
return false;
}
return true;
}
/**
* How many releases to fetch before filtering. Without a filter the limit is
* enough; with one (a release type or either tag list), the newest releases may
* all be filtered out, so the mirror looks further back before applying
* `releaseLimit`.
*/
export function releaseScanLimit(
releaseLimit: number,
releaseType: ReleaseType,
hasTagFilter = false
): number {
return releaseType === "all" && !hasTagFilter
? releaseLimit
: Math.max(releaseLimit, RELEASE_SCAN_WINDOW);
}
/** Per-key sanitizers for the numeric overrides. */
const LIMIT_NORMALIZERS: Record<
MirrorOverrideLimitKey,
@@ -108,6 +404,23 @@ const LIMIT_NORMALIZERS: Record<
releaseAssetLimit: normalizeReleaseAssetLimit,
};
/** Per-key sanitizers for the choice overrides. */
const CHOICE_NORMALIZERS: Record<
MirrorOverrideChoiceKey,
(value: unknown) => ReleaseType | undefined
> = {
releaseType: normalizeReleaseType,
};
/** Per-key sanitizers for the tag-pattern overrides. */
const PATTERN_NORMALIZERS: Record<
MirrorOverridePatternKey,
(value: unknown) => string[] | undefined
> = {
releaseIncludeTags: normalizeReleaseTagPatterns,
releaseExcludeTags: normalizeReleaseTagPatterns,
};
/**
* Whether the release at `index` in the newest-first list gets its assets
* uploaded. `null` means no limit. A release past the limit still gets its
@@ -234,6 +547,9 @@ export function mirrorOptionsToFlags(
mirrorReleases?: boolean;
releaseLimit?: number;
releaseAssetLimit?: number | null;
releaseType?: ReleaseType;
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
mirrorMetadata?: boolean;
metadataComponents?: {
issues?: boolean;
@@ -261,6 +577,11 @@ export function mirrorOptionsToFlags(
normalizeReleaseLimit(mirrorOptions?.releaseLimit) ?? DEFAULT_RELEASE_LIMIT,
releaseAssetLimit:
normalizeReleaseAssetLimit(mirrorOptions?.releaseAssetLimit) ?? null,
releaseType: normalizeReleaseType(mirrorOptions?.releaseType) ?? DEFAULT_RELEASE_TYPE,
releaseIncludeTags:
normalizeReleaseTagPatterns(mirrorOptions?.releaseIncludeTags) ?? [],
releaseExcludeTags:
normalizeReleaseTagPatterns(mirrorOptions?.releaseExcludeTags) ?? [],
};
}
@@ -276,7 +597,7 @@ export const MIRROR_GATING_REASONS = {
/** key -> reason it cannot take effect. Absent key means editable. */
export type MirrorOverrideGating = Partial<
Record<
MirrorOverrideKey | MirrorOverrideLimitKey | OrgSelectionOverrideKey,
MirrorOverrideKey | MirrorOverrideLimitKey | MirrorOverrideChoiceKey | MirrorOverridePatternKey | OrgSelectionOverrideKey,
string
>
>;
@@ -323,6 +644,9 @@ export function getMirrorOverrideGating({
}
gating.releaseLimit = MIRROR_GATING_REASONS.pushTarget;
gating.releaseAssetLimit = MIRROR_GATING_REASONS.pushTarget;
gating.releaseType = MIRROR_GATING_REASONS.pushTarget;
gating.releaseIncludeTags = MIRROR_GATING_REASONS.pushTarget;
gating.releaseExcludeTags = MIRROR_GATING_REASONS.pushTarget;
return gating;
}
@@ -342,24 +666,31 @@ export function getMirrorOverrideGating({
gating.mirrorLabels = MIRROR_GATING_REASONS.labelsFollowIssues;
}
// The release limit rides along with releases: inherit the starred reason
// when that is what disabled releases, otherwise report releases being off.
// Only an explicit `false` counts; an unknown value leaves the field open.
// The release limit and the release filters ride along with releases:
// inherit the starred reason when that is what disabled releases, otherwise
// report releases being off. Only an explicit `false` counts; an unknown
// value leaves the fields open.
if (gating.mirrorReleases) {
gating.releaseLimit = gating.mirrorReleases;
gating.releaseType = gating.mirrorReleases;
} else if (effective.mirrorReleases === false) {
gating.releaseLimit = MIRROR_GATING_REASONS.releasesOff;
gating.releaseType = MIRROR_GATING_REASONS.releasesOff;
}
// The asset limit is read in the same place, so it shares the reason.
// The asset limit and both tag lists are read in the same place, so they
// share the reason.
if (gating.releaseLimit) {
gating.releaseAssetLimit = gating.releaseLimit;
for (const patternKey of MIRROR_OVERRIDE_PATTERN_KEYS) {
gating[patternKey] = gating.releaseLimit;
}
}
return gating;
}
export const MIRROR_OVERRIDE_LABELS: Record<
MirrorOverrideKey | MirrorOverrideLimitKey | OrgSelectionOverrideKey,
MirrorOverrideKey | MirrorOverrideLimitKey | MirrorOverrideChoiceKey | MirrorOverridePatternKey | OrgSelectionOverrideKey,
string
> = {
lfs: "Git LFS files",
@@ -372,6 +703,9 @@ export const MIRROR_OVERRIDE_LABELS: Record<
mirrorMilestones: "Milestones",
releaseLimit: "Release limit",
releaseAssetLimit: "Release asset limit",
releaseType: "Release type",
releaseIncludeTags: "Include tags",
releaseExcludeTags: "Exclude tags",
skipForks: "Skip forks",
};
@@ -415,7 +749,7 @@ export function hasMirrorOverrides(value: unknown): boolean {
/** The keys an overrides object pins, for badges and summaries. */
export function listOverriddenKeys(
value: unknown
): (MirrorOverrideKey | MirrorOverrideLimitKey | OrgSelectionOverrideKey)[] {
): (MirrorOverrideKey | MirrorOverrideLimitKey | MirrorOverrideChoiceKey | MirrorOverridePatternKey | OrgSelectionOverrideKey)[] {
const overrides = parseMirrorOverrides(value);
if (!overrides) return [];
return [
@@ -423,6 +757,12 @@ export function listOverriddenKeys(
...MIRROR_OVERRIDE_LIMIT_KEYS.filter(
(key) => LIMIT_NORMALIZERS[key](overrides[key]) !== undefined
),
...MIRROR_OVERRIDE_CHOICE_KEYS.filter(
(key) => CHOICE_NORMALIZERS[key](overrides[key]) !== undefined
),
...MIRROR_OVERRIDE_PATTERN_KEYS.filter(
(key) => PATTERN_NORMALIZERS[key](overrides[key]) !== undefined
),
...ORG_SELECTION_OVERRIDE_KEYS.filter((key) => overrides[key] != null),
];
}
@@ -448,6 +788,14 @@ export function normalizeMirrorOverrides(
const limit = LIMIT_NORMALIZERS[key](overrides[key]);
if (limit !== undefined) cleaned[key] = limit;
}
for (const key of MIRROR_OVERRIDE_CHOICE_KEYS) {
const choice = CHOICE_NORMALIZERS[key](overrides[key]);
if (choice !== undefined) cleaned[key] = choice;
}
for (const key of MIRROR_OVERRIDE_PATTERN_KEYS) {
const patterns = PATTERN_NORMALIZERS[key](overrides[key]);
if (patterns !== undefined) cleaned[key] = patterns;
}
for (const key of ORG_SELECTION_OVERRIDE_KEYS) {
const flag = overrides[key];
if (typeof flag === "boolean") cleaned[key] = flag;
@@ -462,7 +810,9 @@ export function normalizeMirrorOverrides(
* Precedence is per field, most specific tier wins:
* repository override -> organization override -> global config -> default
* where the default is `false` for flags, DEFAULT_RELEASE_LIMIT for the
* release limit, and `null` (every mirrored release) for the asset limit.
* release limit, `null` (every mirrored release) for the asset limit,
* DEFAULT_RELEASE_TYPE for which releases are candidates, and an empty list for
* the tag patterns (no tag filter).
*
* `starredCodeOnly` is applied last as a hard clamp. It is a "code only, no
* metadata" switch for starred repos, so it forces every metadata flag off
@@ -520,6 +870,31 @@ export function resolveMirrorOptions({
normalizeReleaseAssetLimit(globalConfig?.releaseAssetLimit) ??
null;
// Same precedence for the release type. A value no tier recognizes (a typo
// typed straight into the database, say) falls back to mirroring every
// release rather than to a filter that would hide work from the user.
resolved.releaseType =
normalizeReleaseType(repo?.releaseType) ??
normalizeReleaseType(org?.releaseType) ??
normalizeReleaseType(globalConfig?.releaseType) ??
DEFAULT_RELEASE_TYPE;
// And the same for the tag lists. No tier pins one means no filter, which is
// what every configuration predating the option does. A tier that pins an
// empty or unusable list is not a pin either, so the next tier out still
// applies: clearing the box in the UI means inherit, not "no patterns here".
resolved.releaseIncludeTags =
normalizeReleaseTagPatterns(repo?.releaseIncludeTags) ??
normalizeReleaseTagPatterns(org?.releaseIncludeTags) ??
normalizeReleaseTagPatterns(globalConfig?.releaseIncludeTags) ??
[];
resolved.releaseExcludeTags =
normalizeReleaseTagPatterns(repo?.releaseExcludeTags) ??
normalizeReleaseTagPatterns(org?.releaseExcludeTags) ??
normalizeReleaseTagPatterns(globalConfig?.releaseExcludeTags) ??
[];
// Starred repos with starredCodeOnly mirror code and nothing else. This
// clamp intentionally outranks explicit per-repo overrides: the setting
// exists to stop starred repos from dragging in metadata wholesale.
@@ -0,0 +1,139 @@
/**
* Route tests for GET /api/organizations/:id/preview-repositories.
*
* The organization tag-rule preview offers these repositories in a picker, so
* what matters is that the list is scoped to the caller, limited to the rows
* that belong to the organization, and shaped with a display label.
*
* NOTE: this file replaces @/lib/db with a mock. bun's mock.module is
* process-wide and leaks into other test files, so in the shared test process
* this file registers NOTHING: it re-runs itself in an isolated child process
* (the same harness as organizations/[id].test.ts).
*/
import { describe, test, expect, mock, beforeEach } from "bun:test";
const CHILD_FLAG = "GM_ORG_PREVIEW_REPOS_ISOLATED";
const isChild = !!process.env[CHILD_FLAG];
let orgRows: any[] = [];
let repoRows: any[] = [];
const mockOrganizations = {};
const mockRepositories = {};
if (!isChild) {
test("organization preview-repositories route - isolated child suite", () => {
const res = Bun.spawnSync({
cmd: [process.execPath, "test", import.meta.path],
env: { ...process.env, [CHILD_FLAG]: "1" },
stdout: "pipe",
stderr: "pipe",
});
if (res.exitCode !== 0) {
console.error(res.stdout.toString());
console.error(res.stderr.toString());
}
expect(res.exitCode).toBe(0);
}, 60_000);
}
if (isChild) {
// A select chain that answers both query shapes the route runs: the
// organization lookup ends at limit(), the repository listing passes through
// orderBy() first. The table passed to from() is one of the mocked table
// sentinels, which is how the right row set is chosen.
const mockDb = {
select: mock(() => ({
from: mock((table: any) => ({
where: mock(() => ({
limit: mock(() =>
Promise.resolve(table === mockOrganizations ? orgRows : repoRows)
),
orderBy: mock(() => ({
limit: mock(() =>
Promise.resolve(table === mockRepositories ? repoRows : orgRows)
),
})),
})),
})),
})),
};
// The route pulls in the auth module, which imports every table, so the mock
// has to name them all.
mock.module("@/lib/db", () => ({
db: mockDb,
repositories: mockRepositories,
configs: {},
organizations: mockOrganizations,
users: {},
mirrorJobs: {},
events: {},
accounts: {},
sessions: {},
verificationTokens: {},
verifications: {},
oauthClients: {},
oauthAccessTokens: {},
oauthRefreshTokens: {},
oauthConsents: {},
jwkss: {},
ssoProviders: {},
apikeys: {},
rateLimits: {},
}));
}
const { GET } = isChild
? await import("./preview-repositories")
: { GET: undefined as any };
function call(id?: string) {
return GET({
params: id ? { id } : {},
request: new Request("http://localhost/api/organizations/preview-repositories"),
locals: { session: { userId: "user-1" } },
} as any);
}
beforeEach(() => {
orgRows = [{ name: "octo-org" }];
repoRows = [
{ id: "r1", name: "hello", owner: "octo-org" },
{ id: "r2", name: "world", owner: "octo-org" },
];
});
describe.skipIf(!isChild)("GET /api/organizations/:id/preview-repositories", () => {
test("lists the organization's repositories with a display label", async () => {
const response = await call("org-1");
expect(response.status).toBe(200);
const data = await response.json();
expect(data.success).toBe(true);
expect(data.organization).toBe("octo-org");
expect(data.repositories).toEqual([
{ id: "r1", label: "octo-org/hello" },
{ id: "r2", label: "octo-org/world" },
]);
});
test("answers 404 for an organization the account does not own", async () => {
orgRows = [];
const response = await call("org-1");
expect(response.status).toBe(404);
const data = await response.json();
expect(data.success).toBe(false);
});
test("requires an organization id", async () => {
const response = await call();
expect(response.status).toBe(400);
});
test("returns an empty list when the organization mirrors nothing yet", async () => {
repoRows = [];
const response = await call("org-1");
expect(response.status).toBe(200);
const data = await response.json();
expect(data.repositories).toEqual([]);
});
});
@@ -0,0 +1,72 @@
import type { APIRoute } from "astro";
import { and, asc, eq } from "drizzle-orm";
import { db, organizations, repositories } from "@/lib/db";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
/**
* Cap on how many repositories a single organization override can preview. The
* list only feeds a dropdown, so it is trimmed rather than left to grow without
* bound; the newest names are the ones a user is most likely to reach for.
*/
const MAX_PREVIEW_REPOSITORIES = 200;
/**
* List the repositories an organization override covers, so its release-rule
* dialog can preview one of them.
*
* A global rule has no single repository and a repository override names one,
* but an organization rule fans out over every repository the organization
* mirrors. This route hands the rule editor the repositories it can offer in a
* picker; the per-repository releases then come from the existing
* /api/repositories/:id/releases route once one is chosen.
*
* GET /api/organizations/:id/preview-repositories
*/
export const GET: APIRoute = async ({ params, request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
const orgId = params.id;
if (!orgId) {
return jsonResponse({ data: { success: false, error: "Organization ID is required" }, status: 400 });
}
const [org] = await db
.select({ name: organizations.name })
.from(organizations)
.where(and(eq(organizations.id, orgId), eq(organizations.userId, userId)))
.limit(1);
if (!org) {
return jsonResponse({ data: { success: false, error: "Organization not found" }, status: 404 });
}
// A repository belongs to the organization when its `organization` column
// carries the organization name, the same link the mirror paths use.
const rows = await db
.select({
id: repositories.id,
name: repositories.name,
owner: repositories.owner,
})
.from(repositories)
.where(and(eq(repositories.userId, userId), eq(repositories.organization, org.name)))
.orderBy(asc(repositories.name))
.limit(MAX_PREVIEW_REPOSITORIES);
return jsonResponse({
data: {
success: true,
organization: org.name,
repositories: rows.map((row) => ({
id: row.id,
label: `${row.owner}/${row.name}`,
})),
},
});
} catch (error) {
return createSecureErrorResponse(error, "List organization preview repositories", 500);
}
};
@@ -0,0 +1,245 @@
/**
* Route tests for GET /api/repositories/:id/releases.
*
* The release-rule editor judges its draft filters against whatever this route
* reports, so the two things that matter are the owner the releases are read
* from and the list that comes back. The destination itself is driven through a
* fake fetch, which is what the shared HTTP client uses.
*/
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
import { createMockResponse, mockFetch } from "@/tests/mock-fetch";
const CHILD_FLAG = "GM_RELEASES_ROUTE_ISOLATED";
const isChild = !!process.env[CHILD_FLAG];
if (!isChild) {
test("repositories [id] releases route — isolated child suite", () => {
const res = Bun.spawnSync({
cmd: [process.execPath, "test", import.meta.path],
env: { ...process.env, [CHILD_FLAG]: "1" },
stdout: "pipe",
stderr: "pipe",
});
if (res.exitCode !== 0) {
console.error(res.stdout.toString());
console.error(res.stderr.toString());
}
expect(res.exitCode).toBe(0);
}, 60_000);
}
let repoRows: any[] = [];
let configRows: any[] = [];
const mockRepositories = {};
const mockConfigs = {};
const mockDb = {
select: mock(() => ({
from: mock((table: any) => ({
where: mock(() => ({
limit: mock(() => Promise.resolve(table === mockConfigs ? configRows : repoRows)),
})),
})),
})),
};
// bun's mock.module is process-wide and leaks into other test files, so in the
// shared test process this file registers nothing: it re-runs itself in an
// isolated child process (the same harness as organizations/[id].test.ts).
const mockGetOwner = mock(async () => "mirrors");
if (isChild) {
// The route pulls in the auth module, which imports every table, so the
// mock has to name them all.
mock.module("@/lib/db", () => ({
db: mockDb,
repositories: mockRepositories,
configs: mockConfigs,
organizations: {},
users: {},
mirrorJobs: {},
events: {},
accounts: {},
sessions: {},
verificationTokens: {},
verifications: {},
oauthClients: {},
oauthAccessTokens: {},
oauthRefreshTokens: {},
oauthConsents: {},
jwkss: {},
ssoProviders: {},
apikeys: {},
rateLimits: {},
}));
// Only the owner lookup is needed from the destination client; the releases
// themselves arrive through the fake fetch below.
mock.module("@/lib/gitea", () => ({
getGiteaRepoOwnerAsync: mockGetOwner,
getGiteaRepoOwner: () => "mirrors",
}));
}
const { GET } = isChild
? await import("./releases")
: { GET: undefined as any };
const repoRow = {
id: "r1",
userId: "user-1",
name: "hello",
fullName: "octocat/hello",
owner: "octocat",
organization: null,
isStarred: false,
destinationOrg: null,
mirroredLocation: "mirrors/hello",
status: "mirrored",
destinationProvider: "gitea",
destinationUrl: "https://gitea.example.com",
};
const configRow = {
id: "c1",
userId: "user-1",
isActive: true,
githubConfig: { token: "gh", owner: "octocat", mirrorStrategy: "preserve" },
giteaConfig: {
url: "https://gitea.example.com",
token: "gitea-token",
defaultOwner: "me",
provider: "gitea",
},
};
const realFetch = globalThis.fetch;
let requestedUrls: string[] = [];
/** Serve one response per page, in order. */
function servePages(pages: unknown[]) {
requestedUrls = [];
let index = 0;
globalThis.fetch = mockFetch(async (url: string) => {
requestedUrls.push(url);
const batch = pages[index++];
return batch instanceof Response ? batch : createMockResponse(batch);
}) as unknown as typeof globalThis.fetch;
}
function batch(count: number, offset = 0) {
return Array.from({ length: count }, (_, i) => ({
tag_name: `v${offset + i}`,
prerelease: false,
}));
}
function call(id = "r1") {
return GET({
params: { id },
request: new Request(`http://localhost/api/repositories/${id}/releases`),
locals: { session: { userId: "user-1" } },
} as any);
}
beforeEach(() => {
repoRows = [repoRow];
configRows = [configRow];
mockGetOwner.mockClear();
});
afterEach(() => {
globalThis.fetch = realFetch;
});
describe.skipIf(!isChild)("GET /api/repositories/:id/releases", () => {
test("reads the destination at the place the mirror was put", async () => {
servePages([
[
{ tag_name: "v1.0", prerelease: false },
{ tag_name: "v2.0-rc1", prerelease: true },
],
]);
const response = await call();
expect(response.status).toBe(200);
const data = await response.json();
expect(data.success).toBe(true);
expect(data.destination).toBe("mirrors/hello");
expect(data.releases).toEqual([
{ tag: "v1.0", prerelease: false },
{ tag: "v2.0-rc1", prerelease: true },
]);
expect(requestedUrls[0]).toContain("/api/v1/repos/mirrors/hello/releases");
expect(requestedUrls[0]).toContain("per_page=100");
expect(mockGetOwner).not.toHaveBeenCalled();
});
test("resolves the owner the way a sync would before the mirror is placed", async () => {
repoRows = [{ ...repoRow, mirroredLocation: null }];
servePages([[]]);
const response = await call();
expect(response.status).toBe(200);
const data = await response.json();
expect(data.destination).toBe("mirrors/hello");
expect(data.releases).toEqual([]);
expect(mockGetOwner).toHaveBeenCalledTimes(1);
expect(requestedUrls[0]).toContain("/api/v1/repos/mirrors/hello/releases");
});
test("answers 404 for a repository that is not the account's", async () => {
repoRows = [];
servePages([[]]);
const response = await call();
expect(response.status).toBe(404);
const data = await response.json();
expect(data.success).toBe(false);
expect(requestedUrls).toHaveLength(0);
});
test("refuses before the destination is configured", async () => {
configRows = [
{ ...configRow, giteaConfig: { url: "", token: "", defaultOwner: "" } },
];
servePages([[]]);
const response = await call();
expect(response.status).toBe(400);
expect(requestedUrls).toHaveLength(0);
});
test("says which destination could not be read", async () => {
servePages([
createMockResponse({ message: "not found" }, { ok: false, status: 404, statusText: "Not Found" }),
]);
const response = await call();
expect(response.status).toBe(404);
const data = await response.json();
expect(data.error).toContain("mirrors/hello");
});
test("follows the destination paging until a short page", async () => {
servePages([batch(100), batch(2, 100)]);
const response = await call();
const data = await response.json();
expect(data.releases).toHaveLength(102);
expect(data.truncated).toBe(false);
expect(requestedUrls).toHaveLength(2);
expect(requestedUrls[1]).toContain("page=2");
});
test("stops at the cap and marks the list as shortened", async () => {
servePages([batch(100), batch(100, 100), batch(100, 200), batch(1, 300)]);
const response = await call();
const data = await response.json();
expect(data.releases).toHaveLength(300);
expect(data.truncated).toBe(true);
expect(requestedUrls).toHaveLength(3);
});
});
+136
View File
@@ -0,0 +1,136 @@
import type { APIRoute } from "astro";
import { and, eq } from "drizzle-orm";
import { db, configs, repositories } from "@/lib/db";
import type { Repository } from "@/lib/db/schema";
import type { Config } from "@/types/config";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
import { getGiteaRepoOwnerAsync } from "@/lib/gitea";
import { decryptConfigTokens } from "@/lib/utils/config-encryption";
import { httpGet } from "@/lib/http-client";
/** Gitea's per-page maximum; the cap below is how far the preview looks. */
const PAGE_SIZE = 100;
const MAX_RELEASES = 300;
type DestinationRelease = {
tag: string;
prerelease: boolean | null;
};
/**
* List the releases the destination already holds for one mirrored repository.
*
* The release-rule editor re-runs the draft filters against this list so a user
* can see which published releases a Save would keep and which the next sync
* would delete. The owner is taken from `mirroredLocation` when the mirror has
* already been placed; otherwise it is resolved the way a sync would resolve it.
*
* GET /api/repositories/:id/releases
*/
export const GET: APIRoute = async ({ params, request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
const repoId = params.id;
if (!repoId) {
return jsonResponse({ data: { success: false, error: "Repository ID is required" }, status: 400 });
}
const [row] = await db
.select()
.from(repositories)
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)))
.limit(1);
if (!row) {
return jsonResponse({ data: { success: false, error: "Repository not found" }, status: 404 });
}
const repository = row as Repository;
const [config] = await db
.select()
.from(configs)
.where(and(eq(configs.userId, userId), eq(configs.isActive, true)))
.limit(1);
if (!config) {
return jsonResponse({ data: { success: false, error: "No active configuration found" }, status: 404 });
}
if (!config.giteaConfig?.url || !config.giteaConfig?.token || !config.giteaConfig?.defaultOwner) {
return jsonResponse({
data: { success: false, error: "Configure the destination URL, username and token first" },
status: 400,
});
}
const partial = config as unknown as Partial<Config>;
const location = (repository.mirroredLocation ?? "").trim();
const slash = location.indexOf("/");
let owner = slash > 0 ? location.slice(0, slash) : "";
const repoName = slash > 0 ? location.slice(slash + 1) : repository.name;
if (!owner) {
const fallbackOwner = config.giteaConfig.defaultOwner;
owner = await getGiteaRepoOwnerAsync({ config: partial, repository }).catch(
() => fallbackOwner
);
}
const token = decryptConfigTokens(config as Config).giteaConfig.token;
const base = `${config.giteaConfig.url.replace(/\/+$/, "")}/api/v1/repos/${encodeURIComponent(
owner
)}/${encodeURIComponent(repoName)}/releases`;
const releases: DestinationRelease[] = [];
let truncated = false;
for (let page = 1; ; page++) {
const response = await httpGet<DestinationReleaseGitea[]>(
`${base}?per_page=${PAGE_SIZE}&page=${page}`,
{ Authorization: `token ${token}` }
).catch(() => null);
const batch = response?.data;
if (!Array.isArray(batch)) {
if (page === 1) {
return jsonResponse({
data: {
success: false,
error: `No releases could be read for ${owner}/${repoName} in the destination`,
},
status: 404,
});
}
break;
}
for (const release of batch) {
const tag = typeof release.tag_name === "string" ? release.tag_name : "";
if (!tag) continue;
releases.push({
tag,
prerelease: typeof release.prerelease === "boolean" ? release.prerelease : null,
});
}
if (batch.length < PAGE_SIZE) break;
if (releases.length >= MAX_RELEASES) {
truncated = true;
break;
}
}
return jsonResponse({
data: {
success: true,
destination: `${owner}/${repoName}`,
releases,
truncated,
},
});
} catch (error) {
return createSecureErrorResponse(error, "List destination releases", 500);
}
};
/** Shape Gitea reports for one release; only the fields the preview needs. */
type DestinationReleaseGitea = {
tag_name?: string;
prerelease?: boolean | null;
};
@@ -0,0 +1,114 @@
/**
* Route tests for GET /api/repositories/preview-repositories.
*
* The global tag-rule preview offers these repositories in a picker, so what
* matters is that the list is scoped to the caller and shaped with a display
* label.
*
* NOTE: this file replaces @/lib/db with a mock. bun's mock.module is
* process-wide and leaks into other test files, so in the shared test process
* this file registers NOTHING: it re-runs itself in an isolated child process
* (the same harness as organizations/[id].test.ts).
*/
import { describe, test, expect, mock, beforeEach } from "bun:test";
const CHILD_FLAG = "GM_GLOBAL_PREVIEW_REPOS_ISOLATED";
const isChild = !!process.env[CHILD_FLAG];
let repoRows: any[] = [];
const mockRepositories = {};
if (!isChild) {
test("global preview-repositories route - isolated child suite", () => {
const res = Bun.spawnSync({
cmd: [process.execPath, "test", import.meta.path],
env: { ...process.env, [CHILD_FLAG]: "1" },
stdout: "pipe",
stderr: "pipe",
});
if (res.exitCode !== 0) {
console.error(res.stdout.toString());
console.error(res.stderr.toString());
}
expect(res.exitCode).toBe(0);
}, 60_000);
}
if (isChild) {
const mockDb = {
select: mock(() => ({
from: mock(() => ({
where: mock(() => ({
orderBy: mock(() => ({
limit: mock(() => Promise.resolve(repoRows)),
})),
})),
})),
})),
};
// The route pulls in the auth module, which imports every table, so the mock
// has to name them all.
mock.module("@/lib/db", () => ({
db: mockDb,
repositories: mockRepositories,
configs: {},
organizations: {},
users: {},
mirrorJobs: {},
events: {},
accounts: {},
sessions: {},
verificationTokens: {},
verifications: {},
oauthClients: {},
oauthAccessTokens: {},
oauthRefreshTokens: {},
oauthConsents: {},
jwkss: {},
ssoProviders: {},
apikeys: {},
rateLimits: {},
}));
}
const { GET } = isChild
? await import("./preview-repositories")
: { GET: undefined as any };
function call() {
return GET({
params: {},
request: new Request("http://localhost/api/repositories/preview-repositories"),
locals: { session: { userId: "user-1" } },
} as any);
}
beforeEach(() => {
repoRows = [
{ id: "r1", name: "hello", owner: "octocat" },
{ id: "r2", name: "world", owner: "octo-org" },
];
});
describe.skipIf(!isChild)("GET /api/repositories/preview-repositories", () => {
test("lists the account's repositories with a display label", async () => {
const response = await call();
expect(response.status).toBe(200);
const data = await response.json();
expect(data.success).toBe(true);
expect(data.repositories).toEqual([
{ id: "r1", label: "octocat/hello" },
{ id: "r2", label: "octo-org/world" },
]);
expect(data.truncated).toBe(false);
});
test("returns an empty list when nothing is mirrored yet", async () => {
repoRows = [];
const response = await call();
expect(response.status).toBe(200);
const data = await response.json();
expect(data.repositories).toEqual([]);
});
});
@@ -0,0 +1,54 @@
import type { APIRoute } from "astro";
import { asc, eq } from "drizzle-orm";
import { db, repositories } from "@/lib/db";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
/**
* Cap on how many repositories the global preview picker offers. The list only
* feeds a dropdown, so it is trimmed rather than left to grow without bound.
*/
const MAX_PREVIEW_REPOSITORIES = 200;
/**
* List the account's repositories so the global release-rule dialog can
* preview its draft patterns against one of them.
*
* Global rules hit every repository, so a typo there costs the most; the same
* picker the organization dialog uses applies here, just over the whole
* account. The per-repository releases then come from the existing
* /api/repositories/:id/releases route once one is chosen.
*
* GET /api/repositories/preview-repositories
*/
export const GET: APIRoute = async ({ request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
const rows = await db
.select({
id: repositories.id,
name: repositories.name,
owner: repositories.owner,
})
.from(repositories)
.where(eq(repositories.userId, userId))
.orderBy(asc(repositories.name))
.limit(MAX_PREVIEW_REPOSITORIES);
return jsonResponse({
data: {
success: true,
truncated: rows.length >= MAX_PREVIEW_REPOSITORIES,
repositories: rows.map((row) => ({
id: row.id,
label: `${row.owner}/${row.name}`,
})),
},
});
} catch (error) {
return createSecureErrorResponse(error, "List preview repositories", 500);
}
};
+6
View File
@@ -19,6 +19,12 @@ export const repoStatusEnum = z.enum([
export type RepoStatus = z.infer<typeof repoStatusEnum>;
// Activity entries carry every repository status plus "warning", an alert
// (e.g. release filters emptied a destination) that a repository never has.
export const activityStatusEnum = z.enum([...repoStatusEnum.options, "warning"]);
export type ActivityStatus = z.infer<typeof activityStatusEnum>;
export const repositoryVisibilityEnum = z.enum([
"public",
"private",
+4 -1
View File
@@ -1,4 +1,4 @@
import { type Config as ConfigType } from "@/lib/db/schema";
import { type Config as ConfigType, type ReleaseType } from "@/lib/db/schema";
export type GiteaOrgVisibility = "public" | "private" | "limited";
export type MirrorStrategy = "preserve" | "single-org" | "flat-user" | "mixed";
@@ -86,6 +86,9 @@ export interface MirrorOptions {
mirrorReleases: boolean;
releaseLimit?: number; // Limit number of releases to mirror (default: 10)
releaseAssetLimit?: number | null; // Upload assets only for the newest N releases; null or absent means all, 0 means none
releaseType?: ReleaseType; // "all" | "stable": which releases are mirror candidates before the limit applies
releaseIncludeTags?: string[]; // Glob or /regex/ tag patterns; when set, only matching tags are mirror candidates
releaseExcludeTags?: string[]; // Glob or /regex/ tag patterns; matching tags are skipped, and exclude wins over include
mirrorLFS: boolean; // Mirror Git LFS objects
mirrorMetadata: boolean;
metadataComponents: {
+2 -2
View File
@@ -1,9 +1,9 @@
import type { MembershipRole } from "./organizations";
import type { RepoStatus } from "./Repository";
import type { ActivityStatus } from "./Repository";
export interface FilterParams {
searchTerm?: string;
status?: RepoStatus | ""; // repos, activity and orgs status
status?: ActivityStatus | ""; // repos, activity and orgs status
membershipRole?: MembershipRole | ""; //membership role in orgs
owner?: string; // owner of the repos
organization?: string; // organization of the repos
+7 -1
View File
@@ -36,7 +36,13 @@ If your Gitea instance is reachable at one URL from the Gitea Mirror process and
Below the connection cards, the mirror settings decide what travels with the code: issues, pull requests, labels, milestones, releases, wiki and LFS objects, plus the force push protection strategy. Switches that the current source or destination cannot serve are disabled and say why.
Releases have two numbers next to the switch. **Latest** is how many releases are mirrored per repository, newest first, ten by default. **Assets for latest** is how many of those get their assets uploaded: leave it empty to upload assets for every mirrored release, set it to `0` for release notes and tags only, or set it to a number to keep assets for the newest N releases while older ones keep their notes and tag. The limit never deletes assets that were already uploaded. Both values can be overridden per organization and per repository through the mirror overrides dialog, and the asset limit is `RELEASE_ASSET_LIMIT` in the environment.
Releases have two numbers under the switch. **Latest** is how many releases are mirrored per repository, newest first, ten by default. **Assets for latest** is how many of those get their assets uploaded: leave it empty to upload assets for every mirrored release, set it to `0` for release notes and tags only, or set it to a number to keep assets for the newest N releases while older ones keep their notes and tag. The limit never deletes assets that were already uploaded.
The **Pre-releases** switch under them decides which releases are candidates at all: left on it mirrors every release, turned off it skips everything the source marks as a pre-release. With a filter active the **Latest** count applies to the releases that match, so ten stable releases are kept even when a project ships betas in between, and releases of the other kind are removed from Gitea on the next sync. A release the destination has no pre-release flag for is left alone. The per organization and per repository **Mirror options** dialog offers the same choice under its **Releases** row - **All**, **Stable only**, or **Inherit**, where **Inherit** means the level above - and the release type is `RELEASE_TYPE` in the environment.
**Custom tag rules...** beside that switch opens a dialog with **Include tags** and **Exclude tags**, two comma separated pattern lists that catch the projects shipping release candidates without marking them as pre-releases. The same lists sit under the **Releases** row in the per organization and per repository dialog, where leaving both lists empty means the level above, and in the environment they are `RELEASE_INCLUDE_TAGS` and `RELEASE_EXCLUDE_TAGS`.
Opened from a repository, that dialog also lists the releases Gitea already holds for it and crosses out the ones the draft rules would drop on the next sync, so a pattern can be checked before saving. The organization and global dialogs leave the list out because their rules apply to many repositories at once.
### Automation
@@ -161,6 +161,9 @@ Only used when `DESTINATION_PROVIDER` is `github` or `gitlab`.
| `MIRROR_MILESTONES` | `false` | Mirror milestones. |
| `MIRROR_RELEASES` | `false` | Mirror GitHub releases along with their assets. |
| `RELEASE_LIMIT` | `10` | Maximum releases mirrored per repository, newest first. |
| `RELEASE_TYPE` | `all` | Which releases are candidates: `all` or `stable` (skip what the source marks as a pre-release). With a filter active the limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync. |
| `RELEASE_INCLUDE_TAGS` | empty | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. |
| `RELEASE_EXCLUDE_TAGS` | empty | Comma separated tag patterns in the same syntax; matching releases are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS`. |
| `RELEASE_ASSET_LIMIT` | unset | Upload release assets only for the newest N mirrored releases; older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means notes only. Never deletes assets already uploaded. |
| `MIRROR_WIKI` | `false` | Clone wiki content. |
| `MIRROR_ISSUE_CONCURRENCY` | `3` | Issues processed in parallel. |
@@ -11,7 +11,7 @@ canonical: "https://gitea-mirror.raylabs.io/use-cases/control-release-asset-stor
A GitHub release is a tag, some notes and, usually, a set of attached files: installers, tarballs, container bundles, model weights. The notes and the tag are a few kilobytes. The assets of a project that ships builds for five platforms every two weeks can run to gigabytes per year, and a mirror that copies all of them copies every historical build you will never install.
Gitea Mirror has had a release limit for a while: mirror the newest N releases and skip the rest. Since version 3.30 there is a second, independent limit for assets, so you can keep the notes of every release you mirror and the binaries of only the recent ones.
Gitea Mirror has had a release limit for a while: mirror the newest N releases and skip the rest. Since version 3.30 there is a second, independent limit for assets, so you can keep the notes of every release you mirror and the binaries of only the recent ones. A third control decides which releases count at all, so a project that ships a beta for every feature does not push the stable builds out of the limit.
## The two limits
@@ -24,6 +24,27 @@ On the **Configuration** page, next to the **Releases** switch in the mirror set
Releases are ordered newest first. A release past the asset limit is still created on Gitea with its notes and tag; it just does not get the files. The two limits are independent: "Latest 50, Assets for latest 3" keeps three years of changelog browsable with the last three builds downloadable.
## Which releases count
The **Pre-releases** switch under the two inputs decides which releases are candidates before either limit is applied: leave it on to mirror every release, or turn it off to skip everything the source marks as a pre-release, which is what most people mean by "back up the releases" when a project ships a beta for every feature. The per organization and per repository dialog carries the same choice under its **Releases** row: **All**, **Stable only**, or **Inherit**.
The filter runs first, so **Latest** counts the releases that match. With **Stable only** and **Latest 10** the mirror looks further back than ten releases when it has to, and keeps the ten newest stable ones. Switching the type also prunes: on the next sync the releases of the other kind that are already on Gitea are removed. A release the destination has no pre-release flag for is left alone. Like the two limits, the type can be set per organization and per repository, and **Inherit** in the dialog means the global value.
## Filter by tag name
Plenty of projects ship a release candidate for every feature and never tick the pre-release box, so the type filter cannot tell them apart. **Custom tag rules...** opens a dialog with two lists that can: **Include tags** mirrors only releases whose tag matches, **Exclude tags** skips the ones that do, and an excluded tag wins when both lists name it.
Patterns are comma separated, and each one is either a glob over the whole tag or a regular expression in slashes:
| Pattern | Matches |
|---|---|
| `v1.*` | `v1.0.0`, `v1.4.2`, but not `v2.0.0` |
| `*-rc.*` | `v2.1.0-rc.1`, `2.0-rc2` |
| `v?` | `v1`, `v2`, but not `v10` |
| `/^v\d+\.\d+\.\d+$/` | `v1.2.3`, not `v1.2.3-rc.1` or `nightly-20260101` |
Matching ignores case, and a pattern that cannot compile is ignored rather than failing the run. A tag filter widens the search exactly like the type filter does, so **Latest 1** with `v*` still finds the newest real release behind a wall of nightlies. Releases whose tag no longer matches are removed from Gitea on the next sync, and unlike the type filter a tag pattern can clean up releases the destination has no pre-release flag for, because it only needs the tag name.
## Nothing already uploaded is deleted
The asset limit only decides whether a sync uploads. Lowering it later never removes assets that already reached Gitea. If you mirrored everything for a year and then set the limit to 2, the old files stay until you delete them in Gitea yourself. That is the safe direction for a backup tool, and it means the setting is cheap to experiment with.
@@ -36,11 +57,11 @@ Open **Configuration → Connections**, scroll to the mirror settings and make s
### 2. Override per organization
Some organizations publish big artifacts, others publish none. Open the three-dot menu on an organization row and choose **Mirror options**. Both limits are there next to the switches, with **Inherit** meaning the global value. An organization that ships firmware images can get `0` assets while the rest of your account keeps the default.
Some organizations publish big artifacts, others publish none. Open the three-dot menu on an organization row and choose **Mirror options**. The dialog has two tabs; the release limits and filters are on **Releases**, each with **Inherit** meaning the global value. An organization that ships firmware images can get `0` assets while the rest of your account keeps the default.
### 3. Override per repository
The same dialog exists on repository rows, and a repository value wins over its organization, which wins over the global default. The dialog labels each row with what inherit resolves to right now, so you can see the effective value before you change it.
The same dialog exists on repository rows, and a repository value wins over its organization, which wins over the global default. Each row there is labelled with what inherit resolves to right now, so you can see the effective value before you change it.
### 4. Environment
@@ -50,9 +71,12 @@ For deployments configured from the environment:
MIRROR_RELEASES=true
RELEASE_LIMIT=25
RELEASE_ASSET_LIMIT=3 # unset for all, 0 for notes only
RELEASE_TYPE=stable # all or stable
RELEASE_INCLUDE_TAGS=v* # comma separated globs or /regex/
RELEASE_EXCLUDE_TAGS=*-rc.*
```
`RELEASE_ASSET_LIMIT` seeds the global setting on first boot, the same way the other mirror options do. The [environment variables page](/docs/environment-variables/) lists the rest.
`RELEASE_ASSET_LIMIT`, `RELEASE_TYPE` and the two tag lists seed the global settings on first boot, the same way the other mirror options do. The [environment variables page](/docs/environment-variables/) lists the rest.
## Doing the disk math
@@ -68,7 +92,11 @@ Releases are read from the GitHub API. For GitLab and Gitea or Forgejo sources t
### Does lowering the release limit delete releases on Gitea?
No. Neither limit deletes anything. Releases and assets that already exist stay until you remove them in Gitea.
Yes for the release limit, no for the asset limit. The release limit is a retention setting: on the next sync the mirror removes the oldest releases past the newest N. The asset limit only decides whether a sync uploads, and lowering it never removes assets that already reached Gitea.
### Can I skip beta and release candidate builds?
Turn the **Pre-releases** switch off, or pick **Stable only** in the repository's **Mirror options**. Everything the source marks as a pre-release is then skipped, and the pre-releases already on Gitea are removed on the next sync.
### Do assets sync again if I raise the limit?