feat(mirror): push engine for GitHub and GitLab destinations (#384)

GitHub and GitLab as mirror destinations, both beta. Neither has a usable pull mirror API, so the app keeps a bare clone of each source repository and pushes its branches and tags to the target with force and prune. Target adapters create, archive and delete repositories through each host's API. A dispatcher routes mirror, sync, retry, scheduler, recovery and cleanup by destination; Gitea and Forgejo are unchanged. Migration 0018 records each repository's destination host. Metadata, LFS, wiki and force push protection are disabled with a reason for push targets.

Closes #377
This commit is contained in:
ARUNAVO RAY
2026-09-02 18:23:27 +05:30
committed by GitHub
parent 8f9fc55c8e
commit 38fac54b17
55 changed files with 6385 additions and 223 deletions
+2 -1
View File
@@ -33,6 +33,7 @@ First user signup becomes admin. Configure GitHub and Gitea/Forgejo through the
- 🔁 Mirror public, private, and starred GitHub repos to Gitea/Forgejo
- 🦊 **GitLab (beta) and Gitea/Forgejo sources** - Pick the source in the configuration card. gitlab.com, Codeberg and self hosted instances mirror code, tags, wiki and LFS (issues, pull requests and releases need a GitHub source). See [docs/SOURCE_PROVIDERS.md](docs/SOURCE_PROVIDERS.md)
- 🏛️ **GitHub Enterprise support** - Works with GHES and GHEC with data residency via `GH_API_URL`
- 🚀 **GitHub and GitLab destinations (beta)** - Pick GitHub or GitLab as the destination and the app keeps a bare clone of each repository and pushes its branches and tags there. Code only, and the target is overwritten on every sync. See [docs/PUSH_TARGETS.md](docs/PUSH_TARGETS.md)
- 🏢 Mirror entire organizations with flexible strategies
- 🎯 Custom destination control for repos and organizations
- 📦 **Git LFS support** - Mirror large files with Git LFS
@@ -204,7 +205,7 @@ bun run dev
1. **First Time Setup**
- Navigate to http://localhost:4321
- Create admin account (first user signup)
- Configure GitHub and Gitea/Forgejo connections
- Configure the source and destination connections. The destination is Gitea or Forgejo (pull mirrors), or GitHub or GitLab (push targets, beta)
2. **Mirror Strategies**
- **Preserve Structure**: Maintains GitHub organization structure
+10 -3
View File
@@ -149,19 +149,26 @@ Standard GitHub Enterprise Cloud on `github.com` works with the default — no o
## Gitea Configuration
Settings for the destination Gitea instance.
Settings for the destination. Gitea and Forgejo are pull mirrors; GitHub and GitLab are push targets served by the push engine (see [PUSH_TARGETS.md](PUSH_TARGETS.md)). The `GITEA_*` names apply to every destination kind.
### Connection Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `DESTINATION_PROVIDER` | Whether the destination is Gitea or Forgejo. Same API; changes labels and hints only. | `gitea` | `gitea`, `forgejo` |
| `GITEA_URL` | Gitea instance URL | - | Valid URL |
| `DESTINATION_PROVIDER` | Which host mirrors are created on. Gitea and Forgejo share one API (labels and hints differ). GitHub and GitLab (beta) receive `git push` of branches and tags from this app. | `gitea` | `gitea`, `forgejo`, `github`, `gitlab` |
| `GITEA_URL` | Destination instance URL. Optional for GitHub and GitLab, which default to github.com and gitlab.com; set it for GitHub Enterprise Server or a self hosted GitLab. | - | Valid URL |
| `GITEA_EXTERNAL_URL` | Optional external/browser URL used for dashboard links. API and mirroring still use `GITEA_URL`. | - | Valid URL |
| `GITEA_TOKEN` | Gitea access token | - | - |
| `GITEA_USERNAME` | Gitea username | - | - |
| `GITEA_ORGANIZATION` | Default organization for single-org strategy | `github-mirrors` | Any string |
### Push Targets (GitHub and GitLab destinations)
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_CLONE_DIR` | Where the push engine keeps its bare clones, one per repository. | `<data dir>/mirrors` | Absolute or relative path |
| `PUSH_CONCURRENCY` | How many git fetch or push operations run at once across all repositories. | `2` | 1 to 32 |
### Repository Settings
| Variable | Description | Default | Options |
+85
View File
@@ -0,0 +1,85 @@
# Push targets: GitHub and GitLab as destinations
Gitea and Forgejo are pull mirrors: the app asks them to fetch from the source, and they keep the copy up to date themselves. GitHub has no pull mirror API and GitLab only offers one on paid tiers, so for those two the app runs a push engine instead. This page explains what that engine does, what it needs, and what it does not do.
Push targets are marked **beta** in the destination dropdown.
## How it works
For every repository whose destination is GitHub or GitLab:
1. **A bare clone is kept on disk** under `<data dir>/mirrors/<source host>/<owner>/<name>.git`, next to the SQLite database. The first run clones, later runs fetch with prune, so a branch or tag deleted on the source is dropped from the clone.
2. **The target repository is created when missing.** GitHub: under the account named in the connection card, or under an organization the mirror strategy picks. GitLab: a project under the user, or in a group; a missing top level group is created, a nested group is not.
3. **Branches and tags are pushed** with force and prune. The target always ends up matching the source, including rewritten history and deleted branches. Pull request refs, merge request refs and other host owned namespaces are not pushed; hosts refuse writes to them.
4. **The default branch is set** on the target to match the source, best effort.
5. **Statuses, mirror jobs and activity rows** are the same as for a Gitea mirror, so the dashboard, the scheduler, recovery and cleanup treat both the same way. The repository table shows the destination's icon and links to the target repository.
The scheduler runs the same fetch and push on every scheduled sync. Both steps are idempotent: an interrupted run just runs again.
## The target is overwritten
Every push uses force and prune. Anything committed directly on the target, a branch created there, a tag moved there, is replaced by what the source has on the next sync. Treat a push target as read only, the same way a Gitea pull mirror is.
## What is not mirrored
Push targets receive **branches and tags only**. The following are not available for them and show as disabled in the settings:
- issues, pull requests, labels, milestones
- releases and release assets (tags do travel)
- wiki
- LFS objects
- force push protection and the backup strategies
Several destinations per repository, and pulling anything back from the target, are out of scope. Mirroring is one way.
## Credentials
Tokens never appear in a URL and are never written to disk. Each git invocation gets an inline credential helper that reads the token from an environment variable set on that process only: the source token for the fetch, the target token for the push. The remote URL stored in the bare clone is the plain source URL.
Token scopes:
| Target | Scopes | Notes |
|--------|--------|-------|
| GitHub | `repo`, `workflow` | `workflow` is needed as soon as a repository contains GitHub Actions files, or the push is rejected. Add `delete_repo` only if cleanup should delete repositories on GitHub. |
| GitLab | `api`, `write_repository` | Pushes use the `oauth2` user name with the token, as GitLab documents. |
GitHub organizations cannot be created through the API. Create the organization first and make the token's user a member with permission to create repositories.
## Disk usage and concurrency
A bare clone takes roughly the size of the repository's git objects, so the data directory grows by about the size of everything you mirror this way. Clones are removed when the repository row is deleted, and by cleanup when it deletes an orphaned repository.
| Variable | Default | Meaning |
|----------|---------|---------|
| `MIRROR_CLONE_DIR` | `<data dir>/mirrors` | Where the bare clones live. |
| `PUSH_CONCURRENCY` | `2` | How many git fetch or push operations run at the same time across all repositories. API calls are cheap, a clone or a mirror push is not. |
One git process per repository at a time is enforced with a lock file next to the clone. A lock older than an hour belongs to a run that died and is taken over; a half written clone (no `HEAD`) is removed and cloned again.
## Large first pushes
GitHub limits the size of a single push. When the single push of all refs is refused, the engine retries in batches of 50 refs, branches first, then tags, followed by a prune. If a batch is still refused, the mirror fails with the git error in the repository's status.
## Cleanup
When the source deletes a repository, the cleanup service applies the configured action to the target through its API: archive on GitHub (`archived: true`) or GitLab (`/archive`), or delete. Deleting on GitHub needs the `delete_repo` scope; without it the cleanup run fails with a message naming the scope and the repository stays. On gitlab.com and instances with delayed deletion, a deleted project is renamed and kept for the configured number of days before it disappears. The bare clone is removed with the row.
The **Reconcile with destination** action compares a Gitea or Forgejo server with the database; it is not available for push targets, which have no server side mirror state to reconcile.
## Switching destinations
Once anything has been mirrored the destination is locked and changing it asks for confirmation, as for Gitea. Rows remember which destination they were mirrored to; a row mirrored to one host is refused on another until it is removed and mirrored again, so a token is never sent to the wrong host.
## Environment variables
```env
DESTINATION_PROVIDER=github # or gitlab
GITEA_URL=https://github.com # optional for github.com and gitlab.com; set for GHES or self hosted GitLab
GITEA_USERNAME=your-account
GITEA_TOKEN=your-token
GITEA_ORGANIZATION=your-org # optional, used by the single-org strategy
MIRROR_CLONE_DIR=/data/mirrors # optional
PUSH_CONCURRENCY=2 # optional
```
The `GITEA_*` names are kept for every destination kind so an existing deployment only has to change `DESTINATION_PROVIDER`. See [ENVIRONMENT_VARIABLES.md](ENVIRONMENT_VARIABLES.md).
+3
View File
@@ -19,6 +19,9 @@ This folder contains engineering and operations references for the open-source G
- **[SSO_TESTING.md](./SSO_TESTING.md)** – Recipes for local and staging SSO testing (Google, Keycloak, mock providers).
- **[API.md](./API.md)** – Create API keys and call the app from scripts, CI pipelines and workflow tools.
### Destinations
- **[PUSH_TARGETS.md](./PUSH_TARGETS.md)** – GitHub and GitLab as destinations: how the push engine works, token scopes, disk usage, what is not mirrored.
If you are looking for customer-facing playbooks, see the MDX use cases under `www/src/pages/use-cases/`.
## Quick start for local development
+7
View File
@@ -0,0 +1,7 @@
ALTER TABLE `repositories` ADD `destination_provider` text DEFAULT 'gitea' NOT NULL;--> statement-breakpoint
ALTER TABLE `repositories` ADD `destination_url` text DEFAULT '' NOT NULL;--> statement-breakpoint
UPDATE `repositories` SET
`destination_provider` = COALESCE((SELECT json_extract(`configs`.`gitea_config`, '$.provider') FROM `configs` WHERE `configs`.`id` = `repositories`.`config_id`), 'gitea'),
`destination_url` = COALESCE((SELECT json_extract(`configs`.`gitea_config`, '$.url') FROM `configs` WHERE `configs`.`id` = `repositories`.`config_id`), '')
WHERE `destination_url` = '';--> statement-breakpoint
UPDATE `repositories` SET `destination_provider` = 'gitea' WHERE `destination_provider` NOT IN ('gitea', 'forgejo', 'github', 'gitlab');
File diff suppressed because it is too large Load Diff
+7
View File
@@ -127,6 +127,13 @@
"when": 1788339447860,
"tag": "0017_api_keys",
"breakpoints": true
},
{
"idx": 18,
"version": "6",
"when": 1788350408052,
"tag": "0018_destination_columns",
"breakpoints": true
}
]
}
+78
View File
@@ -583,6 +583,80 @@ function verify0017Migration(db: any) {
assert(remaining.count === 0, "Expected deleting a user to cascade to their API keys");
}
function seedPre0018Database(db: any) {
// Migrations 0000-0017 have run. Rows carry no destination columns yet;
// seed one account whose gitea_config names a Forgejo server and one with
// an empty config, so the backfill can be verified against both.
db.run("INSERT INTO users (id, email, username, name) VALUES ('u-dest', 'dest@example.com', 'dest', 'Destination User')");
db.run(
"INSERT INTO configs (id, user_id, name, is_active, github_config, gitea_config, schedule_config, cleanup_config) " +
"VALUES ('cfg-forgejo', 'u-dest', 'Default', 1, '{}', '{\"provider\":\"forgejo\",\"url\":\"https://forge.example.com\"}', '{}', '{}')",
);
db.run(
"INSERT INTO configs (id, user_id, name, is_active, github_config, gitea_config, schedule_config, cleanup_config) " +
"VALUES ('cfg-empty', 'u-dest', 'Empty', 0, '{}', '{}', '{}', '{}')",
);
db.run(
"INSERT INTO repositories (id, user_id, config_id, name, full_name, normalized_full_name, url, clone_url, owner, default_branch) " +
"VALUES ('repo-forgejo', 'u-dest', 'cfg-forgejo', 'tool', 'dest/tool', 'dest/tool', 'https://github.com/dest/tool', 'https://github.com/dest/tool.git', 'dest', 'main')",
);
db.run(
"INSERT INTO repositories (id, user_id, config_id, name, full_name, normalized_full_name, url, clone_url, owner, default_branch) " +
"VALUES ('repo-empty', 'u-dest', 'cfg-empty', 'lib', 'dest/lib', 'dest/lib', 'https://github.com/dest/lib', 'https://github.com/dest/lib.git', 'dest', 'main')",
);
}
function verify0018Migration(db: any) {
const cols = db
.query("PRAGMA table_info(repositories)")
.all() as Array<{ name: string; notnull: number; dflt_value: string | null }>;
for (const [name, expectedDefault] of [
["destination_provider", "'gitea'"],
["destination_url", "''"],
] as const) {
const col = cols.find((c) => c.name === name);
assert(col, `Expected repositories.${name} column to exist`);
assert(col.notnull === 1, `Expected repositories.${name} to be NOT NULL`);
assert(
col.dflt_value === expectedDefault,
`Expected repositories.${name} default ${expectedDefault}, got ${col.dflt_value}`,
);
}
// Existing rows take the destination of the account that imported them.
const forgejo = db
.query("SELECT destination_provider, destination_url FROM repositories WHERE id = 'repo-forgejo'")
.get() as { destination_provider: string; destination_url: string } | null;
assert(
forgejo?.destination_provider === "forgejo" && forgejo.destination_url === "https://forge.example.com",
`Expected the Forgejo account's row to be backfilled from its config, got ${JSON.stringify(forgejo)}`,
);
// An account without a destination yet leaves the URL empty, which the
// guard treats as "not recorded", never as a different host.
const empty = db
.query("SELECT destination_provider, destination_url FROM repositories WHERE id = 'repo-empty'")
.get() as { destination_provider: string; destination_url: string } | null;
assert(
empty?.destination_provider === "gitea" && empty.destination_url === "",
`Expected an account without a destination to default to gitea with no URL, got ${JSON.stringify(empty)}`,
);
// New rows can record a push target.
db.run(
"INSERT INTO repositories (id, user_id, config_id, name, full_name, normalized_full_name, url, clone_url, owner, default_branch, destination_provider, destination_url) " +
"VALUES ('repo-github-target', 'u-dest', 'cfg-forgejo', 'app', 'dest/app', 'dest/app', 'https://gitlab.com/dest/app', 'https://gitlab.com/dest/app.git', 'dest', 'main', 'github', 'https://github.com')",
);
const inserted = db
.query("SELECT destination_provider, destination_url FROM repositories WHERE id = 'repo-github-target'")
.get() as { destination_provider: string; destination_url: string } | null;
assert(
inserted?.destination_provider === "github" && inserted.destination_url === "https://github.com",
`Expected a GitHub destination to round-trip, got ${JSON.stringify(inserted)}`,
);
}
const MIGRATION_0012_TIMESTAMP = 1774062000000;
const MIGRATION_0013_TIMESTAMP = 1780377747526;
@@ -778,6 +852,10 @@ const latestUpgradeFixtures: Record<string, UpgradeFixture> = {
seed: seedPre0017Database,
verify: verify0017Migration,
},
"0018_destination_columns": {
seed: seedPre0018Database,
verify: verify0018Migration,
},
};
function lintMigrations(selectedMigrations: Migration[]) {
+1
View File
@@ -809,6 +809,7 @@ export function ConfigTabs() {
<GitHubMirrorSettings
part="content"
githubConfig={config.githubConfig}
destinationProvider={config.giteaConfig.provider}
mirrorOptions={config.mirrorOptions}
advancedOptions={config.advancedOptions}
onGitHubConfigChange={newConfig => {
+29 -10
View File
@@ -39,7 +39,8 @@ import {
UserX,
X
} from "lucide-react";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy } from "@/types/config";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy, DestinationProvider } from "@/types/config";
import { DESTINATION_PROVIDER_LABELS, isPushDestinationKind } from "@/lib/destination-kinds";
import {
Select,
SelectContent,
@@ -74,6 +75,8 @@ import {
interface GitHubMirrorSettingsProps {
githubConfig: GitHubConfig;
mirrorOptions: MirrorOptions;
/** The configured destination. GitHub and GitLab targets receive code only. */
destinationProvider?: DestinationProvider;
advancedOptions: AdvancedOptions;
onGitHubConfigChange: (config: GitHubConfig) => void;
onMirrorOptionsChange: (options: MirrorOptions) => void;
@@ -89,12 +92,18 @@ export function GitHubMirrorSettings({
onGitHubConfigChange,
onMirrorOptionsChange,
onAdvancedOptionsChange,
destinationProvider,
part = "both",
}: GitHubMirrorSettingsProps) {
// A GitHub or GitLab destination is pushed by the engine: branches and
// tags only, so every content switch is off and disabled for it.
const isPushTarget = isPushDestinationKind(destinationProvider ?? "gitea");
const destinationLabel = DESTINATION_PROVIDER_LABELS[destinationProvider ?? "gitea"];
const pushTargetReason = `Not available for ${destinationLabel} destinations: only branches and tags are pushed`;
// Star lists, issues, pull requests, releases, labels and milestones all
// read the GitHub API, so they are only offered for GitHub sources.
const sourceProvider = githubConfig.provider ?? "github";
const isGithubSource = sourceProvider === "github";
const isGithubSource = sourceProvider === "github" && !isPushTarget;
const sourceLabel = SOURCE_PROVIDER_LABELS[sourceProvider];
const orgNoun = SOURCE_PROVIDER_ORG_NOUNS[sourceProvider];
const [starListsOpen, setStarListsOpen] = React.useState(false);
@@ -858,9 +867,11 @@ export function GitHubMirrorSettings({
<StatusFooterItem
icon={Info}
label={
isGithubSource
? "Pull requests are mirrored as issues due to Gitea API limits"
: "Issues, pull requests and releases are mirrored for GitHub sources only"
isPushTarget
? `${destinationLabel} destinations receive branches and tags only`
: isGithubSource
? "Pull requests are mirrored as issues due to Gitea API limits"
: "Issues, pull requests and releases are mirrored for GitHub sources only"
}
/>
}
@@ -883,7 +894,9 @@ export function GitHubMirrorSettings({
icon={Tag}
label="Releases & tags"
description={
!isGithubSource
isPushTarget
? pushTargetReason
: !isGithubSource
? "Release assets need a GitHub source. Tags always come with the code"
: mirrorOptions.mirrorReleases
? "Includes tags. Leave the assets box empty to upload assets for every mirrored release, or set it to 0 for release notes only"
@@ -944,8 +957,11 @@ export function GitHubMirrorSettings({
<SwitchRow
icon={HardDrive}
label="Git LFS objects"
description="Requires LFS enabled on your Gitea server and Git v2.1.2+"
description={
isPushTarget ? pushTargetReason : "Requires LFS enabled on your Gitea server and Git v2.1.2+"
}
badge="BETA"
disabled={isPushTarget}
checked={mirrorOptions.mirrorLFS}
onCheckedChange={(checked) => handleMirrorChange('mirrorLFS', checked)}
/>
@@ -954,10 +970,13 @@ export function GitHubMirrorSettings({
icon={FileText}
label="Repository metadata"
description={
isGithubSource
? "Issues, pull requests, labels, milestones and wiki"
: "Wiki only. Issues, pull requests, labels and milestones need a GitHub source"
isPushTarget
? pushTargetReason
: isGithubSource
? "Issues, pull requests, labels, milestones and wiki"
: "Wiki only. Issues, pull requests, labels and milestones need a GitHub source"
}
disabled={isPushTarget}
checked={mirrorOptions.mirrorMetadata}
onCheckedChange={(checked) => handleMirrorChange('mirrorMetadata', checked)}
extra={mirrorOptions.mirrorMetadata ? metadataPopover : undefined}
+113 -38
View File
@@ -1,8 +1,8 @@
import React, { useState, useEffect } from "react";
import { Button } from "@/components/ui/button";
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
import { Activity, AlertTriangle, Building2, Info, PlugZap } from "lucide-react";
import { SiForgejo, SiGitea } from "react-icons/si";
import { Activity, AlertTriangle, Building2, Info, PlugZap, UploadCloud } from "lucide-react";
import { SiForgejo, SiGitea, SiGithub, SiGitlab } from "react-icons/si";
import {
Select,
SelectContent,
@@ -10,7 +10,13 @@ import {
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { DESTINATION_PROVIDER_LABELS } from "@/lib/destination-kinds";
import {
DESTINATION_PROVIDER_DEFAULT_URLS,
DESTINATION_PROVIDER_LABELS,
DESTINATION_PROVIDER_ORG_NOUNS,
PUSH_DESTINATION_TOKEN_SCOPES,
isPushDestinationKind,
} from "@/lib/destination-kinds";
import { HostLockNotice } from "./HostLockNotice";
import { giteaApi, type GiteaServerInfo } from "@/lib/api";
import type {
@@ -50,11 +56,18 @@ const destinationProviders: {
value: DestinationProvider;
label: string;
icon: React.ComponentType<{ className?: string }>;
/** Short pill shown next to the option, e.g. BETA. */
badge?: string;
}[] = [
{ value: "gitea", label: DESTINATION_PROVIDER_LABELS.gitea, icon: SiGitea },
{ value: "forgejo", label: DESTINATION_PROVIDER_LABELS.forgejo, icon: SiForgejo },
{ value: "github", label: DESTINATION_PROVIDER_LABELS.github, icon: SiGithub, badge: "BETA" },
{ value: "gitlab", label: DESTINATION_PROVIDER_LABELS.gitlab, icon: SiGitlab, badge: "BETA" },
];
/** The public instance URLs a hosted target is prefilled with. */
const HOSTED_DEFAULT_URLS = new Set(Object.values(DESTINATION_PROVIDER_DEFAULT_URLS));
export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, githubUsername, destinationLock, part = "connection" }: GiteaConfigFormProps) {
const [isLoading, setIsLoading] = useState(false);
const [destinationUnlocked, setDestinationUnlocked] = useState(false);
@@ -63,9 +76,21 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
const providerMeta =
destinationProviders.find((option) => option.value === provider) ?? destinationProviders[0];
const providerLabel = providerMeta.label;
const isPushTarget = isPushDestinationKind(provider);
const orgNoun = DESTINATION_PROVIDER_ORG_NOUNS[provider];
const handleProviderChange = (value: string) => {
const newConfig: GiteaConfig = { ...config, provider: value as DestinationProvider };
const nextProvider = value as DestinationProvider;
const newConfig: GiteaConfig = { ...config, provider: nextProvider };
// A hosted target is prefilled with its public instance; moving away
// from one clears that prefill so a Gitea URL is asked for again.
const hostedDefault = DESTINATION_PROVIDER_DEFAULT_URLS[nextProvider];
const currentUrl = (config.url || "").trim();
if (hostedDefault && (!currentUrl || HOSTED_DEFAULT_URLS.has(currentUrl))) {
newConfig.url = hostedDefault;
} else if (!hostedDefault && HOSTED_DEFAULT_URLS.has(currentUrl)) {
newConfig.url = "";
}
setConfig(newConfig);
if (onAutoSave) {
onAutoSave(newConfig, { confirmDestinationChange: destinationUnlocked });
@@ -170,7 +195,7 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
};
const testConnection = async () => {
if (!config.url || !config.token) {
if (!config.token || (!isPushTarget && !config.url)) {
toast.error(`${providerLabel} URL and token are required to test the connection`);
return;
}
@@ -178,14 +203,17 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
setIsLoading(true);
try {
const result = await giteaApi.testConnection(config.url, config.token);
const result = await giteaApi.testConnection(config.url, config.token, {
provider,
username: isPushTarget ? config.username : undefined,
});
if (result.success) {
setServerInfo(result.serverInfo ?? null);
toast.success("Successfully connected to Gitea!");
toast.success(`Successfully connected to ${providerLabel}!`);
} else {
setServerInfo(null);
toast.error(
"Failed to connect to Gitea. Please check your URL and token."
result.message || `Failed to connect to ${providerLabel}. Please check your URL and token.`
);
}
} catch (error) {
@@ -198,6 +226,13 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
}
};
const serverInfoLabel = (info: GiteaServerInfo): string => {
if (info.type === "github" || info.type === "gitlab") {
return `Connected to ${info.version}`;
}
return `${info.type === "forgejo" ? "Forgejo" : "Gitea"} ${info.version} detected`;
};
if (part === "connection") {
return (
<SettingsCard
@@ -213,7 +248,7 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
variant="outline"
size="sm"
onClick={testConnection}
disabled={isLoading || !config.url || !config.token}
disabled={isLoading || !config.token || (!isPushTarget && !config.url)}
>
<PlugZap className="mr-1.5 h-3.5 w-3.5" />
{isLoading ? "Testing..." : "Test"}
@@ -222,10 +257,7 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
}
footer={
serverInfo ? (
<StatusFooterItem
icon={Info}
label={`${serverInfo.type === "forgejo" ? "Forgejo" : "Gitea"} ${serverInfo.version} detected`}
/>
<StatusFooterItem icon={Info} label={serverInfoLabel(serverInfo)} />
) : undefined
}
>
@@ -273,13 +305,20 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
<span className="flex items-center gap-2">
<option.icon className="h-3.5 w-3.5" />
{option.label}
{option.badge && (
<span className="rounded-full bg-muted px-1.5 py-0.5 text-[9px] font-semibold tracking-wider text-muted-foreground">
{option.badge}
</span>
)}
</span>
</SelectItem>
))}
</SelectContent>
</Select>
<p className="text-[11px] text-muted-foreground/80">
Where mirrors are created. Gitea and Forgejo share the same API.
{isPushTarget
? `${providerLabel} has no pull mirror API, so this app keeps a bare clone of each source repository and pushes its branches and tags there.`
: "Where mirrors are created. Gitea and Forgejo share the same API."}
</p>
{destinationLock?.locked && (
<HostLockNotice
@@ -299,12 +338,26 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
)}
</div>
{isPushTarget && (
<Alert>
<UploadCloud className="h-4 w-4" />
<AlertTitle>Push target (beta)</AlertTitle>
<AlertDescription>
<p>
Branches and tags are pushed with force and prune, so the {providerLabel} copy always matches the
source and is overwritten on every sync. Issues, pull requests, releases, wiki and LFS objects are
not mirrored. Bare clones live in the data directory next to the database.
</p>
</AlertDescription>
</Alert>
)}
<div className="space-y-1.5">
<Label
htmlFor="gitea-username"
className="text-xs font-medium text-muted-foreground"
>
Username
{isPushTarget ? "Account" : "Username"}
</Label>
<Input
id="gitea-username"
@@ -315,6 +368,11 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
placeholder={`Your ${providerLabel} username`}
required
/>
{isPushTarget && (
<p className="text-[11px] text-muted-foreground/80">
{`The user the token belongs to. Repositories go under this account, or under the ${orgNoun} the mirror strategy picks.`}
</p>
)}
</div>
<div className="space-y-1.5">
@@ -322,7 +380,7 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
htmlFor="gitea-url"
className="text-xs font-medium text-muted-foreground"
>
Server URL
{isPushTarget ? "Instance URL" : "Server URL"}
</Label>
<Input
id="gitea-url"
@@ -330,31 +388,42 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
type="url"
value={config.url}
onChange={handleChange}
placeholder={`https://your-${provider}-instance.com`}
placeholder={
DESTINATION_PROVIDER_DEFAULT_URLS[provider] ?? `https://your-${provider}-instance.com`
}
disabled={destinationLocked}
required
required={!isPushTarget}
/>
{isPushTarget && (
<p className="text-[11px] text-muted-foreground/80">
{provider === "github"
? "github.com, or the URL of your GitHub Enterprise Server"
: "gitlab.com, or the URL of your self hosted GitLab"}
</p>
)}
</div>
<div className="space-y-1.5">
<Label
htmlFor="gitea-external-url"
className="text-xs font-medium text-muted-foreground"
>
External URL (optional)
</Label>
<Input
id="gitea-external-url"
name="externalUrl"
type="url"
value={config.externalUrl || ""}
onChange={handleChange}
placeholder="https://gitea.example.com"
/>
<p className="text-[11px] text-muted-foreground/80">
Dashboard links only, syncing always uses the server URL
</p>
</div>
{!isPushTarget && (
<div className="space-y-1.5">
<Label
htmlFor="gitea-external-url"
className="text-xs font-medium text-muted-foreground"
>
External URL (optional)
</Label>
<Input
id="gitea-external-url"
name="externalUrl"
type="url"
value={config.externalUrl || ""}
onChange={handleChange}
placeholder="https://gitea.example.com"
/>
<p className="text-[11px] text-muted-foreground/80">
Dashboard links only, syncing always uses the server URL
</p>
</div>
)}
<div className="space-y-1.5">
<Label
@@ -373,7 +442,13 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
required
/>
<p className="text-[11px] text-muted-foreground/80">
{`Create one in ${providerLabel} under Settings → Applications`}
{isPushTarget
? `A personal access token with the scopes ${PUSH_DESTINATION_TOKEN_SCOPES[provider].join(", ")}. ${
provider === "github"
? "GitHub rejects pushes that add workflow files unless the token has the workflow scope."
: "Groups the token cannot see are created as new top level groups."
}`
: `Create one in ${providerLabel} under Settings → Applications`}
</p>
</div>
</CardSection>
@@ -87,6 +87,8 @@ export interface MirrorOverridesDialogProps {
/** Repository-only: drives the starred-code-only gating. */
isStarred?: boolean;
starredCodeOnly?: boolean;
/** The configured destination kind; GitHub and GitLab targets disable every toggle. */
destinationProvider?: string | null;
onSave: (overrides: MirrorOverrides | null) => Promise<void>;
}
@@ -101,6 +103,7 @@ export function MirrorOverridesDialog({
inheritedLoading = false,
isStarred,
starredCodeOnly,
destinationProvider,
onSave,
}: MirrorOverridesDialogProps) {
const [draft, setDraft] = useState<Record<MirrorOverrideKey, TriState>>(
@@ -143,8 +146,9 @@ export function MirrorOverridesDialog({
isStarred,
starredCodeOnly,
effective,
destinationProvider,
}),
[targetKind, isStarred, starredCodeOnly, effective]
[targetKind, isStarred, starredCodeOnly, effective, destinationProvider]
);
const pinnedReleaseLimit = parseReleaseLimitDraft(releaseLimitDraft);
+9 -6
View File
@@ -1,8 +1,9 @@
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import { SourceIcon, repositorySourceInfo } from "@/components/source/SourceIcon";
import { DestinationIcon, destinationInfo } from "@/components/destination/DestinationIcon";
import { Button } from "@/components/ui/button";
import { GitFork } from "lucide-react";
import { SiGithub, SiGitea } from "react-icons/si";
import { SiGithub } from "react-icons/si";
import type { Repository } from "@/lib/db/schema";
import { buildGiteaWebUrl } from "@/lib/gitea-url";
import { useGiteaConfig } from "@/hooks/useGiteaConfig";
@@ -15,9 +16,11 @@ interface RepositoryListProps {
export function RepositoryList({ repositories }: RepositoryListProps) {
const { giteaConfig } = useGiteaConfig();
// Helper function to construct Gitea repository URL
const destination = destinationInfo(giteaConfig);
// Helper function to construct the destination repository URL
const getGiteaRepoUrl = (repository: Repository): string | null => {
// Only provide Gitea links for repositories that have been or are being mirrored
// Only provide links for repositories that have been or are being mirrored
const validStatuses = ['mirroring', 'mirrored', 'syncing', 'synced'];
if (!validStatuses.includes(repository.status)) {
return null;
@@ -113,14 +116,14 @@ export function RepositoryList({ repositories }: RepositoryListProps) {
href={giteaUrl}
target="_blank"
rel="noopener noreferrer"
title="View on Gitea"
title={`View on ${destination.label}`}
>
<SiGitea className="h-4 w-4" />
<DestinationIcon provider={destination.provider} className="h-4 w-4" />
</a>
</Button>
) : (
<Button variant="ghost" size="icon" className="h-8 w-8" disabled title="Not mirrored yet">
<SiGitea className="h-4 w-4" />
<DestinationIcon provider={destination.provider} className="h-4 w-4" />
</Button>
);
})()}
@@ -0,0 +1,36 @@
import { SiForgejo, SiGitea, SiGithub, SiGitlab } from "react-icons/si";
import {
DESTINATION_PROVIDER_LABELS,
isPushDestinationKind,
normalizeDestinationProviderKind,
type DestinationProviderKind,
} from "@/lib/destination-kinds";
const icons: Record<DestinationProviderKind, React.ComponentType<{ className?: string }>> = {
gitea: SiGitea,
forgejo: SiForgejo,
github: SiGithub,
gitlab: SiGitlab,
};
/** Brand icon for the host mirrors are created on. */
export function DestinationIcon({
provider,
className,
}: {
provider: DestinationProviderKind | string | null | undefined;
className?: string;
}) {
const Icon = icons[normalizeDestinationProviderKind(provider)] ?? SiGitea;
return <Icon className={className} />;
}
/** Kind, label and transport for a configured destination. */
export function destinationInfo(config: { provider?: string | null } | null | undefined): {
provider: DestinationProviderKind;
label: string;
isPushTarget: boolean;
} {
const provider = normalizeDestinationProviderKind(config?.provider);
return { provider, label: DESTINATION_PROVIDER_LABELS[provider], isPushTarget: isPushDestinationKind(provider) };
}
@@ -743,6 +743,7 @@ export function OrganizationList({
targetKind="organization"
targetName={overridesTarget?.name ?? ""}
value={overridesTarget?.mirrorOverrides ?? null}
destinationProvider={giteaConfig?.provider}
inheritedFrom={mirrorOptionsToFlags(mirrorOptions)}
inheritedLabel="global settings"
onSave={async (overrides) => {
+20 -15
View File
@@ -1,5 +1,6 @@
import { useEffect, useMemo, useRef, useState } from "react";
import { SourceIcon, repositorySourceInfo } from "@/components/source/SourceIcon";
import { DestinationIcon, destinationInfo } from "@/components/destination/DestinationIcon";
import {
getCoreRowModel,
getFilteredRowModel,
@@ -11,7 +12,7 @@ import {
} from "@tanstack/react-table";
import { useVirtualizer } from "@tanstack/react-virtual";
import { FlipHorizontal, GitFork, MoreVertical, RefreshCw, RotateCcw, Star, Lock, Ban, Check, ChevronDown, SlidersHorizontal, Trash2, X } from "lucide-react";
import { SiGithub, SiGitea } from "react-icons/si";
import { SiGithub } from "react-icons/si";
import type { MirrorOverrides, Repository } from "@/lib/db/schema";
import { repoStatusEnum, type RepoStatus } from "@/types/Repository";
import { Button } from "@/components/ui/button";
@@ -230,9 +231,12 @@ export default function RepositoryTable({
}
};
// Helper function to construct Gitea repository URL
// The configured destination: drives the icon, label and tooltips of the target links.
const destination = destinationInfo(giteaConfig);
// Helper function to construct the destination repository URL
const getGiteaRepoUrl = (repository: Repository): string | null => {
// Only provide Gitea links for repositories that have been or are being mirrored
// Only provide links for repositories that have been or are being mirrored
const validStatuses = ['mirroring', 'mirrored', 'syncing', 'synced', 'archived'];
if (!validStatuses.includes(repository.status)) {
return null;
@@ -662,17 +666,17 @@ export default function RepositoryTable({
href={giteaUrl}
target="_blank"
rel="noopener noreferrer"
title="View on Gitea"
title={`View on ${destination.label}`}
className="flex items-center justify-center gap-2"
>
<SiGitea className="h-4 w-4 flex-shrink-0" />
<span className="text-xs">Gitea</span>
<DestinationIcon provider={destination.provider} className="h-4 w-4 flex-shrink-0" />
<span className="text-xs">{destination.label}</span>
</a>
</Button>
) : (
<Button variant="outline" size="default" disabled className="flex-1 h-10 min-w-0">
<SiGitea className="h-4 w-4" />
<span className="text-xs ml-2">Gitea</span>
<DestinationIcon provider={destination.provider} className="h-4 w-4" />
<span className="text-xs ml-2">{destination.label}</span>
</Button>
)}
</div>
@@ -1100,17 +1104,17 @@ export default function RepositoryTable({
// Determine tooltip based on status and configuration
let tooltip: string;
if (!giteaConfig?.url) {
tooltip = "Gitea not configured";
tooltip = `${destination.label} not configured`;
} else if (repo.status === 'imported') {
tooltip = "Repository not yet mirrored to Gitea";
tooltip = `Repository not yet mirrored to ${destination.label}`;
} else if (repo.status === 'failed') {
tooltip = "Repository mirroring failed";
} else if (repo.status === 'mirroring') {
tooltip = "Repository is being mirrored to Gitea";
tooltip = `Repository is being mirrored to ${destination.label}`;
} else if (giteaUrl) {
tooltip = "View on Gitea";
tooltip = `View on ${destination.label}`;
} else {
tooltip = "Gitea repository not available";
tooltip = `${destination.label} repository not available`;
}
return giteaUrl ? (
@@ -1121,12 +1125,12 @@ export default function RepositoryTable({
rel="noopener noreferrer"
title={tooltip}
>
<SiGitea className="h-4 w-4" />
<DestinationIcon provider={destination.provider} className="h-4 w-4" />
</a>
</Button>
) : (
<Button variant="ghost" size="icon" disabled title={tooltip}>
<SiGitea className="h-4 w-4" />
<DestinationIcon provider={destination.provider} className="h-4 w-4" />
</Button>
);
})()}
@@ -1162,6 +1166,7 @@ export default function RepositoryTable({
inheritedLoading={orgOverridesLoading}
isStarred={!!overridesTarget?.isStarred}
starredCodeOnly={!!advancedOptions?.starredCodeOnly}
destinationProvider={giteaConfig?.provider}
inheritedLabel={
overridesTarget?.organization
? "organization or global settings"
+9 -4
View File
@@ -1,5 +1,5 @@
import { withBase } from "@/lib/base-path";
import type { SourceProvider } from "@/types/config";
import type { SourceProvider, DestinationProvider } from "@/types/config";
// Base API URL
const API_BASE = withBase("/api");
@@ -93,19 +93,24 @@ export const githubApi = {
// Gitea API
export interface GiteaServerInfo {
type: "forgejo" | "gitea";
type: "forgejo" | "gitea" | "github" | "gitlab";
version: string;
raw: string;
hasMirrorCredBug: boolean;
}
export const giteaApi = {
testConnection: (url: string, token: string) =>
/** Test the destination connection. The route serves Gitea, Forgejo, GitHub and GitLab. */
testConnection: (
url: string,
token: string,
options: { provider?: DestinationProvider; username?: string } = {}
) =>
apiRequest<{ success: boolean; serverInfo?: GiteaServerInfo; message?: string }>(
"/gitea/test-connection",
{
method: "POST",
body: JSON.stringify({ url, token }),
body: JSON.stringify({ url, token, ...options }),
}
),
};
+13 -2
View File
@@ -3,6 +3,7 @@ import {
SOURCE_PROVIDER_KINDS,
type SourceProviderKind,
} from "../source-providers/kinds";
import type { DestinationProviderKind } from "../destination-kinds";
import { sqliteTable, text, integer, index, uniqueIndex } from "drizzle-orm/sqlite-core";
import { sql } from "drizzle-orm";
@@ -89,8 +90,9 @@ export type MirrorOverrides = z.infer<typeof mirrorOverridesSchema>;
export const giteaConfigSchema = z.object({
url: z.url(),
// Gitea or Forgejo. Same API; only labels and hints differ.
provider: z.enum(["gitea", "forgejo"]).default("gitea"),
// Gitea and Forgejo are pull mirrors (same API); GitHub and GitLab are
// push targets served by src/lib/push-engine.
provider: z.enum(["gitea", "forgejo", "github", "gitlab"]).default("gitea"),
externalUrl: z.url().optional(),
token: z.string(),
defaultOwner: z.string(),
@@ -240,6 +242,8 @@ export const repositorySchema = z.object({
owner: z.string(),
sourceProvider: z.string().optional(),
sourceUrl: z.string().optional(),
destinationProvider: z.string().optional(),
destinationUrl: z.string().optional(),
organization: z.string().optional().nullable(),
mirroredLocation: z.string().default(""),
isPrivate: z.boolean().default(false),
@@ -467,6 +471,13 @@ export const repositories = sqliteTable("repositories", {
.notNull()
.default("github"),
sourceUrl: text("source_url").notNull().default("https://github.com"),
// Where the mirror lives: the destination kind and base URL recorded when
// the row was imported or mirrored. An empty URL means "not recorded yet".
destinationProvider: text("destination_provider")
.$type<DestinationProviderKind>()
.notNull()
.default("gitea"),
destinationUrl: text("destination_url").notNull().default(""),
owner: text("owner").notNull(),
organization: text("organization"),
mirroredLocation: text("mirrored_location").default(""),
+107
View File
@@ -0,0 +1,107 @@
/**
* The destination a config points at, server side.
*
* destination-kinds.ts stays import free for the client; this module adds
* the pieces that need the config: the decrypted token, the account name,
* and the guard that stops a repository from being pushed to a host other
* than the one it was mirrored to.
*/
import type { Config, Repository } from "@/lib/db/schema";
import { getDecryptedGiteaToken } from "@/lib/utils/config-encryption";
import {
describeDestination,
getRepositoryDestination,
isPushDestinationKind,
isRepositoryOnConfiguredDestination,
normalizeDestinationBaseUrl,
normalizeDestinationProviderKind,
type DestinationProviderKind,
type PushDestinationKind,
type RepositoryDestination,
type RepositoryDestinationFields,
} from "./destination-kinds";
import { createPushTarget, type PushTarget } from "./push-engine/targets";
export interface DestinationConnection extends RepositoryDestination {
/** The account the token belongs to on the destination. */
username: string;
token: string;
organization?: string;
}
/** The destination kind a config selects, defaulting to Gitea. */
export function resolveDestinationKind(config: Partial<Config> | null | undefined): DestinationProviderKind {
return normalizeDestinationProviderKind(config?.giteaConfig?.provider);
}
/** True when mirrors for this config go through the push engine. */
export function usesPushEngine(config: Partial<Config> | null | undefined): boolean {
return isPushDestinationKind(resolveDestinationKind(config));
}
export function resolvePushDestinationKind(config: Partial<Config>): PushDestinationKind {
const kind = resolveDestinationKind(config);
if (!isPushDestinationKind(kind)) {
throw new Error(`The configured destination (${kind}) is not a push target.`);
}
return kind;
}
/** The destination without the token, for comparisons and messages. */
export function resolveDestinationIdentity(config: Partial<Config> | null | undefined): RepositoryDestination {
const provider = resolveDestinationKind(config);
return { provider, url: normalizeDestinationBaseUrl(config?.giteaConfig?.url, provider) };
}
/** Build the connection for a config. The token is passed in decrypted or read from the config. */
export function resolveDestinationConnection(
config: Partial<Config>,
{ token }: { token?: string } = {}
): DestinationConnection {
const identity = resolveDestinationIdentity(config);
return {
...identity,
username: config.giteaConfig?.defaultOwner ?? "",
token: token ?? (config.giteaConfig?.token ? getDecryptedGiteaToken(config as Config) : ""),
organization: config.giteaConfig?.organization || undefined,
};
}
/** The push target adapter for a config whose destination is GitHub or GitLab. */
export function createPushTargetFromConfig(config: Partial<Config>): PushTarget {
const kind = resolvePushDestinationKind(config);
const connection = resolveDestinationConnection(config);
if (!connection.token) {
throw new Error(`A ${kind === "gitlab" ? "GitLab" : "GitHub"} token is required to push mirrors.`);
}
return createPushTarget(kind, {
url: connection.url,
username: connection.username,
token: connection.token,
});
}
/**
* Refuse to touch a repository whose mirror lives on a different host.
*
* The mirror image of assertRepositoryMatchesConfiguredSource: after the
* destination was switched, a row still pointing at the old server must not
* be pushed to, synced on or deleted from the new one with the new token.
*/
export function assertRepositoryMatchesConfiguredDestination({
repository,
config,
}: {
repository: Pick<Repository, "fullName"> & RepositoryDestinationFields;
config: Partial<Config>;
}): void {
const destination = resolveDestinationIdentity(config);
if (isRepositoryOnConfiguredDestination(repository, destination)) return;
throw new Error(
`Repository ${repository.fullName} is mirrored to ${describeDestination(
getRepositoryDestination(repository)
)} but the configured destination is ${describeDestination(destination)}. ` +
"Switch the destination back, or remove the repository and mirror it again."
);
}
+76
View File
@@ -0,0 +1,76 @@
import { describe, expect, test } from "bun:test";
import {
describeDestination,
destinationHostOf,
getRepositoryDestination,
isDestinationProviderKind,
isPushDestinationKind,
isRepositoryOnConfiguredDestination,
normalizeDestinationBaseUrl,
normalizeDestinationProviderKind,
} from "./destination-kinds";
describe("destination kinds", () => {
test("recognizes the four kinds and maps codeberg to forgejo", () => {
expect(isDestinationProviderKind("gitea")).toBe(true);
expect(isDestinationProviderKind("forgejo")).toBe(true);
expect(isDestinationProviderKind("github")).toBe(true);
expect(isDestinationProviderKind("gitlab")).toBe(true);
expect(isDestinationProviderKind("bitbucket")).toBe(false);
expect(normalizeDestinationProviderKind("codeberg")).toBe("forgejo");
expect(normalizeDestinationProviderKind(undefined)).toBe("gitea");
expect(normalizeDestinationProviderKind("nonsense")).toBe("gitea");
});
test("only GitHub and GitLab are push targets", () => {
expect(isPushDestinationKind("github")).toBe(true);
expect(isPushDestinationKind("gitlab")).toBe(true);
expect(isPushDestinationKind("gitea")).toBe(false);
expect(isPushDestinationKind("forgejo")).toBe(false);
});
test("normalizes base URLs and falls back to the public instance for hosted targets", () => {
expect(normalizeDestinationBaseUrl("", "github")).toBe("https://github.com");
expect(normalizeDestinationBaseUrl(undefined, "gitlab")).toBe("https://gitlab.com");
expect(normalizeDestinationBaseUrl("", "gitea")).toBe("");
expect(normalizeDestinationBaseUrl("GitLab.example.com/", "gitlab")).toBe("https://gitlab.example.com");
expect(normalizeDestinationBaseUrl("https://ghe.example.com/api/", "github")).toBe("https://ghe.example.com/api");
expect(normalizeDestinationBaseUrl("ftp://nope", "github")).toBe("https://github.com");
expect(destinationHostOf("https://GitHub.com/x")).toBe("github.com");
});
test("rows without a recorded URL are accepted on any destination", () => {
const destination = { provider: "github" as const, url: "https://github.com" };
expect(isRepositoryOnConfiguredDestination({}, destination)).toBe(true);
expect(isRepositoryOnConfiguredDestination({ destinationProvider: "gitea", destinationUrl: "" }, destination)).toBe(true);
});
test("rows on a different host or a different transport are refused", () => {
const github = { provider: "github" as const, url: "https://github.com" };
expect(
isRepositoryOnConfiguredDestination({ destinationProvider: "github", destinationUrl: "https://github.com/" }, github)
).toBe(true);
expect(
isRepositoryOnConfiguredDestination({ destinationProvider: "gitlab", destinationUrl: "https://gitlab.com" }, github)
).toBe(false);
expect(
isRepositoryOnConfiguredDestination(
{ destinationProvider: "gitea", destinationUrl: "https://gitea.example.com" },
{ provider: "forgejo", url: "https://gitea.example.com" }
)
).toBe(true);
expect(
isRepositoryOnConfiguredDestination(
{ destinationProvider: "gitea", destinationUrl: "https://gitea.example.com" },
{ provider: "gitea", url: "https://other.example.com" }
)
).toBe(false);
});
test("describes a stored destination", () => {
expect(describeDestination(getRepositoryDestination({ destinationProvider: "gitlab", destinationUrl: "https://gitlab.com" }))).toBe(
"GitLab (https://gitlab.com)"
);
expect(describeDestination(getRepositoryDestination({}))).toBe("Gitea (no URL)");
});
});
+119 -4
View File
@@ -1,17 +1,49 @@
/**
* The hosts Gitea Mirror can mirror into. Gitea and Forgejo share the same
* API, so this only changes labels and hints; kept free of imports so client
* components can use it.
* The hosts Gitea Mirror can mirror into.
*
* Gitea and Forgejo share the same API and are pull mirrors: the destination
* fetches from the source itself. GitHub and GitLab have no usable pull
* mirror API, so they are push targets: this app keeps a bare clone and
* pushes it with `git push --mirror` (see src/lib/push-engine).
*
* Kept free of imports so client components can use it.
*/
export const DESTINATION_PROVIDER_KINDS = ["gitea", "forgejo"] as const;
export const DESTINATION_PROVIDER_KINDS = ["gitea", "forgejo", "github", "gitlab"] as const;
export type DestinationProviderKind = (typeof DESTINATION_PROVIDER_KINDS)[number];
/** Destinations that receive mirrors through the push engine. */
export const PUSH_DESTINATION_KINDS = ["github", "gitlab"] as const;
export type PushDestinationKind = (typeof PUSH_DESTINATION_KINDS)[number];
export const DEFAULT_DESTINATION_PROVIDER: DestinationProviderKind = "gitea";
export const DESTINATION_PROVIDER_LABELS: Record<DestinationProviderKind, string> = {
gitea: "Gitea",
forgejo: "Forgejo",
github: "GitHub",
gitlab: "GitLab",
};
/** Base URL used when the config has no URL for a hosted destination. */
export const DESTINATION_PROVIDER_DEFAULT_URLS: Partial<Record<DestinationProviderKind, string>> = {
github: "https://github.com",
gitlab: "https://gitlab.com",
};
/** What the host calls a group of repositories owned by a team. */
export const DESTINATION_PROVIDER_ORG_NOUNS: Record<DestinationProviderKind, string> = {
gitea: "organization",
forgejo: "organization",
github: "organization",
gitlab: "group",
};
/** Token scopes a push target needs, for the settings hint. */
export const PUSH_DESTINATION_TOKEN_SCOPES: Record<PushDestinationKind, string[]> = {
github: ["repo", "workflow", "delete_repo (only for cleanup deletes)"],
gitlab: ["api", "write_repository"],
};
export function isDestinationProviderKind(value: unknown): value is DestinationProviderKind {
@@ -21,9 +53,92 @@ export function isDestinationProviderKind(value: unknown): value is DestinationP
);
}
export function isPushDestinationKind(value: unknown): value is PushDestinationKind {
return (
typeof value === "string" && (PUSH_DESTINATION_KINDS as readonly string[]).includes(value)
);
}
/** Coerce an untrusted value to a destination kind, defaulting to Gitea. Codeberg is Forgejo. */
export function normalizeDestinationProviderKind(value: unknown): DestinationProviderKind {
if (isDestinationProviderKind(value)) return value;
if (value === "codeberg") return "forgejo";
return DEFAULT_DESTINATION_PROVIDER;
}
/**
* Normalize a destination URL: trim, add https:// when the scheme is
* missing, lowercase the host, drop trailing slashes. A hosted push target
* with an empty URL falls back to its public instance; Gitea and Forgejo
* have no default and stay empty.
*/
export function normalizeDestinationBaseUrl(
raw: string | null | undefined,
provider: DestinationProviderKind
): string {
const fallback = DESTINATION_PROVIDER_DEFAULT_URLS[provider] ?? "";
const trimmed = typeof raw === "string" ? raw.trim() : "";
if (!trimmed) return fallback;
const withScheme = /^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed) ? trimmed : `https://${trimmed}`;
let url: URL;
try {
url = new URL(withScheme);
} catch {
return fallback;
}
if (url.protocol !== "http:" && url.protocol !== "https:") {
return fallback;
}
const path = url.pathname.replace(/\/+$/, "");
return `${url.protocol}//${url.host}${path}`;
}
/** The two columns a stored repository carries about where its mirror lives. */
export interface RepositoryDestinationFields {
destinationProvider?: string | null;
destinationUrl?: string | null;
}
export interface RepositoryDestination {
provider: DestinationProviderKind;
url: string;
}
/** Where a stored repository's mirror lives. Rows from before the column default to Gitea. */
export function getRepositoryDestination(repo: RepositoryDestinationFields): RepositoryDestination {
const provider = normalizeDestinationProviderKind(repo.destinationProvider);
return { provider, url: normalizeDestinationBaseUrl(repo.destinationUrl, provider) };
}
/**
* True when the row's mirror lives on the configured destination. A row
* without a recorded URL predates the column (or was never mirrored) and is
* accepted: the Gitea path then finds or creates the mirror as before.
*/
export function isRepositoryOnConfiguredDestination(
repo: RepositoryDestinationFields,
destination: RepositoryDestination
): boolean {
const recordedUrl = typeof repo.destinationUrl === "string" ? repo.destinationUrl.trim() : "";
if (!recordedUrl) return true;
const recorded = getRepositoryDestination(repo);
return (
isPushDestinationKind(recorded.provider) === isPushDestinationKind(destination.provider) &&
recorded.url.toLowerCase() === destination.url.toLowerCase()
);
}
export function describeDestination(destination: RepositoryDestination): string {
return `${DESTINATION_PROVIDER_LABELS[destination.provider]} (${destination.url || "no URL"})`;
}
/** Host name of a destination URL, for display and for the clone directory layout. */
export function destinationHostOf(url: string): string {
try {
return new URL(url).host.toLowerCase();
} catch {
return "";
}
}
+9 -3
View File
@@ -8,7 +8,7 @@ import { eq, and, sql } from 'drizzle-orm';
import { v4 as uuidv4 } from 'uuid';
import { encrypt } from '@/lib/utils/encryption';
import { isSourceProviderKind, type SourceProviderKind } from '@/lib/source-providers/kinds';
import { isDestinationProviderKind, type DestinationProviderKind } from '@/lib/destination-kinds';
import { DESTINATION_PROVIDER_DEFAULT_URLS, isDestinationProviderKind, type DestinationProviderKind } from '@/lib/destination-kinds';
import { hasDestinationChanged, hasSourceChanged, loadConfigLocks } from '@/lib/config-locks';
import { normalizeReleaseAssetLimit } from '@/lib/utils/mirror-overrides';
@@ -153,7 +153,8 @@ function parseEnvConfig(): EnvConfig {
mirrorStrategy: process.env.MIRROR_STRATEGY as 'preserve' | 'single-org' | 'flat-user' | 'mixed',
},
gitea: {
// DESTINATION_PROVIDER picks Gitea or Forgejo (labels only, same API).
// DESTINATION_PROVIDER picks Gitea or Forgejo (pull mirrors, same API),
// or GitHub or GitLab (push targets served by the push engine).
provider: isDestinationProviderKind(process.env.DESTINATION_PROVIDER)
? process.env.DESTINATION_PROVIDER
: undefined,
@@ -349,7 +350,12 @@ export async function initializeConfigFromEnv(): Promise<void> {
// Build Gitea config
const giteaConfig = {
url: envConfig.gitea.url || existingConfig?.[0]?.giteaConfig?.url || '',
url:
envConfig.gitea.url ||
existingConfig?.[0]?.giteaConfig?.url ||
// GitHub and GitLab targets default to their public instance.
DESTINATION_PROVIDER_DEFAULT_URLS[envConfig.gitea.provider || existingConfig?.[0]?.giteaConfig?.provider || 'gitea'] ||
'',
provider: envConfig.gitea.provider || existingConfig?.[0]?.giteaConfig?.provider || 'gitea',
externalUrl: envConfig.gitea.externalUrl || existingConfig?.[0]?.giteaConfig?.externalUrl || undefined,
token: envConfig.gitea.token ? encrypt(envConfig.gitea.token) : existingConfig?.[0]?.giteaConfig?.token || '',
+6
View File
@@ -12,6 +12,7 @@ import type { Repository } from "./db/schema";
import type { Octokit } from "@octokit/rest";
import { createGitHubClient } from "./github";
import { createMirrorJob } from "./helpers";
import { assertRepositoryMatchesConfiguredDestination, usesPushEngine } from "./destination-connection";
import { decryptConfigTokens } from "./utils/config-encryption";
import { httpPost, httpGet, httpPatch, HttpError } from "./http-client";
import { db, repositories } from "./db";
@@ -344,6 +345,11 @@ export async function syncGiteaRepoEnhanced({
const decryptedConfig = decryptConfigTokens(config as Config);
if (usesPushEngine(config)) {
throw new Error("The configured destination is a push target; syncing goes through the push engine.");
}
assertRepositoryMatchesConfiguredDestination({ repository, config });
console.log(`[Sync] Starting sync for repository ${repository.name}`);
// Mark repo as "syncing" in DB
+25 -3
View File
@@ -1,3 +1,4 @@
import { assertRepositoryMatchesConfiguredDestination, usesPushEngine } from "./destination-connection";
import {
repoStatusEnum,
type RepositoryVisibility,
@@ -651,6 +652,11 @@ export const mirrorGithubRepoToGitea = async ({
// Never send another host's token: the repository must come from the
// source that is configured now.
assertRepositoryMatchesConfiguredSource({ repository, config });
if (usesPushEngine(config)) {
throw new Error("The configured destination is a push target; the Gitea mirror path cannot serve it.");
}
// Nor to a different destination than the one the row was mirrored to.
assertRepositoryMatchesConfiguredDestination({ repository, config });
// Get the correct owner based on the strategy (with organization overrides)
let repoOwner = await getGiteaRepoOwnerAsync({ config, repository });
@@ -1499,6 +1505,11 @@ export async function mirrorGitHubRepoToGiteaOrg({
// Never send another host's token: the repository must come from the
// source that is configured now.
assertRepositoryMatchesConfiguredSource({ repository, config });
if (usesPushEngine(config)) {
throw new Error("The configured destination is a push target; the Gitea mirror path cannot serve it.");
}
// Nor to a different destination than the one the row was mirrored to.
assertRepositoryMatchesConfiguredDestination({ repository, config });
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
@@ -2090,11 +2101,16 @@ export async function mirrorGitHubOrgToGitea({
let giteaOrgId: number | undefined;
let targetOrgName: string;
// GitHub and GitLab destinations are pushed by the engine, which creates
// the target repository (and a GitLab group) itself; no Gitea
// organization is created for them.
const pushTransport = usesPushEngine(config);
// Determine the target organization based on strategy
if (organization.destinationOrg) {
// Organization-level override takes precedence over the strategy
targetOrgName = organization.destinationOrg;
giteaOrgId = await getOrCreateGiteaOrg({
giteaOrgId = pushTransport ? undefined : await getOrCreateGiteaOrg({
orgId: organization.id,
orgName: targetOrgName,
config,
@@ -2103,7 +2119,7 @@ export async function mirrorGitHubOrgToGitea({
} else if (mirrorStrategy === "single-org" && config.giteaConfig?.organization) {
// For single-org strategy, use the configured destination organization
targetOrgName = config.giteaConfig.organization || config.giteaConfig.defaultOwner;
giteaOrgId = await getOrCreateGiteaOrg({
giteaOrgId = pushTransport ? undefined : await getOrCreateGiteaOrg({
orgId: organization.id,
orgName: targetOrgName,
config,
@@ -2112,7 +2128,7 @@ export async function mirrorGitHubOrgToGitea({
} else if (mirrorStrategy === "preserve") {
// For preserve strategy, create/use an org with the same name as GitHub
targetOrgName = organization.name;
giteaOrgId = await getOrCreateGiteaOrg({
giteaOrgId = pushTransport ? undefined : await getOrCreateGiteaOrg({
orgId: organization.id,
orgName: targetOrgName,
config,
@@ -2169,6 +2185,12 @@ export async function mirrorGitHubOrgToGitea({
`Starting mirror for repository: ${repo.name} from GitHub org ${organization.name}`
);
if (pushTransport) {
const { pushMirrorRepository } = await import("./push-engine/mirror");
await pushMirrorRepository({ config, repository: repoData });
return repo;
}
// Resolve per repo with the canonical precedence
const owner = await getGiteaRepoOwnerAsync({ config, repository: repoData });
+72
View File
@@ -0,0 +1,72 @@
/**
* Which path a destination takes: Gitea and Forgejo through the pull
* mirror functions, GitHub and GitLab through the push engine.
*/
import { describe, expect, mock, test } from "bun:test";
import { createMirrorDispatcher, resolveMirrorTransport, type MirrorDispatchDeps } from "./mirror-dispatch";
function deps() {
return {
push: mock<MirrorDispatchDeps["push"]>(async () => "push"),
giteaMirror: mock<MirrorDispatchDeps["giteaMirror"]>(async () => "gitea-mirror"),
giteaOrgMirror: mock<MirrorDispatchDeps["giteaOrgMirror"]>(async () => "gitea-org-mirror"),
giteaSync: mock<MirrorDispatchDeps["giteaSync"]>(async () => "gitea-sync"),
} satisfies MirrorDispatchDeps;
}
const repository = { id: "r1", name: "tool", fullName: "acme/tool", owner: "acme" } as any;
function configFor(provider: string | undefined) {
return { userId: "u1", giteaConfig: { provider, url: "https://example.com", token: "t", defaultOwner: "me" } } as any;
}
describe("resolveMirrorTransport", () => {
test("pull for Gitea and Forgejo, push for GitHub and GitLab, pull when unset", () => {
expect(resolveMirrorTransport(configFor("gitea"))).toBe("pull");
expect(resolveMirrorTransport(configFor("forgejo"))).toBe("pull");
expect(resolveMirrorTransport(configFor("github"))).toBe("push");
expect(resolveMirrorTransport(configFor("gitlab"))).toBe("push");
expect(resolveMirrorTransport(configFor(undefined))).toBe("pull");
expect(resolveMirrorTransport(undefined)).toBe("pull");
});
});
describe("mirror dispatcher", () => {
test("mirror to Gitea goes to the user path without an organization", async () => {
const d = deps();
const dispatcher = createMirrorDispatcher(d);
await dispatcher.mirror({ config: configFor("gitea"), octokit: null, repository });
expect(d.giteaMirror).toHaveBeenCalledTimes(1);
expect(d.giteaOrgMirror).not.toHaveBeenCalled();
expect(d.push).not.toHaveBeenCalled();
});
test("mirror to Forgejo with an organization goes to the org path", async () => {
const d = deps();
const dispatcher = createMirrorDispatcher(d);
await dispatcher.mirror({ config: configFor("forgejo"), octokit: null, repository, orgName: "acme", giteaOrgId: 3 });
expect(d.giteaOrgMirror).toHaveBeenCalledTimes(1);
expect(d.giteaOrgMirror.mock.calls[0][0]).toMatchObject({ orgName: "acme", giteaOrgId: 3 });
expect(d.giteaMirror).not.toHaveBeenCalled();
});
test("mirror to GitHub goes to the push engine even when an organization is given", async () => {
const d = deps();
const dispatcher = createMirrorDispatcher(d);
await dispatcher.mirror({ config: configFor("github"), octokit: null, repository, orgName: "acme" });
expect(d.push).toHaveBeenCalledTimes(1);
expect(d.push.mock.calls[0][0]).toMatchObject({ repository, mode: "mirror" });
expect(d.giteaMirror).not.toHaveBeenCalled();
expect(d.giteaOrgMirror).not.toHaveBeenCalled();
});
test("sync routes by destination too", async () => {
const d = deps();
const dispatcher = createMirrorDispatcher(d);
await dispatcher.sync({ config: configFor("gitlab"), repository });
await dispatcher.sync({ config: configFor("gitea"), repository });
expect(d.push).toHaveBeenCalledTimes(1);
expect(d.push.mock.calls[0][0]).toMatchObject({ mode: "sync" });
expect(d.giteaSync).toHaveBeenCalledTimes(1);
});
});
+100
View File
@@ -0,0 +1,100 @@
/**
* One entry point per operation, whatever the destination.
*
* Gitea and Forgejo are pull mirrors: the existing functions in gitea.ts
* ask the destination to fetch. GitHub and GitLab are push targets served
* by the push engine. Every route, the scheduler and recovery go through
* here so the choice is made in one place, and tests can inject the
* implementations to assert which path a destination takes.
*/
import type { Octokit } from "@octokit/rest";
import type { Config, Repository } from "@/lib/db/schema";
import { usesPushEngine } from "@/lib/destination-connection";
export interface MirrorRepositoryArgs {
config: Partial<Config>;
octokit: Octokit | null;
repository: Repository;
/** Set when the caller already resolved the destination organization. */
orgName?: string;
/** Set when the caller already created the Gitea organization. */
giteaOrgId?: number;
}
export interface SyncRepositoryArgs {
config: Partial<Config>;
repository: Repository;
}
export interface MirrorDispatchDeps {
push: (args: { config: Partial<Config>; repository: Repository; mode: "mirror" | "sync" }) => Promise<unknown>;
giteaMirror: (args: { config: Partial<Config>; octokit: Octokit | null; repository: Repository }) => Promise<unknown>;
giteaOrgMirror: (args: {
config: Partial<Config>;
octokit: Octokit | null;
repository: Repository;
orgName: string;
giteaOrgId?: number;
}) => Promise<unknown>;
giteaSync: (args: SyncRepositoryArgs) => Promise<unknown>;
}
export type MirrorTransport = "pull" | "push";
/** Which transport a config's destination uses. */
export function resolveMirrorTransport(config: Partial<Config> | null | undefined): MirrorTransport {
return usesPushEngine(config) ? "push" : "pull";
}
export function createMirrorDispatcher(deps: MirrorDispatchDeps) {
return {
async mirror({ config, octokit, repository, orgName, giteaOrgId }: MirrorRepositoryArgs): Promise<unknown> {
if (resolveMirrorTransport(config) === "push") {
return deps.push({ config, repository, mode: "mirror" });
}
if (orgName) {
return deps.giteaOrgMirror({ config, octokit, repository, orgName, giteaOrgId });
}
return deps.giteaMirror({ config, octokit, repository });
},
async sync({ config, repository }: SyncRepositoryArgs): Promise<unknown> {
if (resolveMirrorTransport(config) === "push") {
return deps.push({ config, repository, mode: "sync" });
}
return deps.giteaSync({ config, repository });
},
};
}
/** The production dispatcher. Imports are lazy so this module stays cheap to load and cycle free. */
const defaultDispatcher = createMirrorDispatcher({
push: async (args) => {
const { pushMirrorRepository } = await import("@/lib/push-engine/mirror");
return pushMirrorRepository(args);
},
giteaMirror: async ({ config, octokit, repository }) => {
const { mirrorGithubRepoToGitea } = await import("@/lib/gitea");
return mirrorGithubRepoToGitea({ config, octokit, repository });
},
giteaOrgMirror: async ({ config, octokit, repository, orgName, giteaOrgId }) => {
const gitea = await import("@/lib/gitea");
if (giteaOrgId !== undefined) {
return gitea.mirrorGitHubRepoToGiteaOrg({ config, octokit, repository, orgName, giteaOrgId });
}
return gitea.mirrorGitHubOrgRepoToGiteaOrg({ config, octokit, repository, orgName });
},
giteaSync: async ({ config, repository }) => {
const { syncGiteaRepo } = await import("@/lib/gitea");
return syncGiteaRepo({ config, repository });
},
});
/** Create or update the mirror of one repository on the configured destination. */
export function mirrorRepositoryToDestination(args: MirrorRepositoryArgs): Promise<unknown> {
return defaultDispatcher.mirror(args);
}
/** Bring an existing mirror up to date on the configured destination. */
export function syncRepositoryOnDestination(args: SyncRepositoryArgs): Promise<unknown> {
return defaultDispatcher.sync(args);
}
+61
View File
@@ -0,0 +1,61 @@
/**
* What the cleanup service and the delete routes need from the push engine:
* archive or delete the repository on the target, and drop the bare clone
* once its database row is gone.
*/
import type { Config, Repository } from "@/lib/db/schema";
import { createPushTargetFromConfig } from "@/lib/destination-connection";
import { getGiteaRepoOwnerAsync } from "@/lib/gitea";
import { removeClone } from "./engine";
import { parseMirroredLocation } from "./mirror";
import { clonePathForRepository, type CloneRepositoryFields } from "./paths";
type TargetRepositoryRow = CloneRepositoryFields & {
mirroredLocation?: string | null;
fullName?: string;
} & Partial<Repository>;
/** Where a row's mirror lives on the target: the recorded location, else the strategy's answer. */
export async function resolvePushTargetLocation(
config: Partial<Config>,
repository: TargetRepositoryRow
): Promise<{ owner: string; name: string }> {
const recorded = parseMirroredLocation(repository.mirroredLocation);
if (recorded) return recorded;
const owner = await getGiteaRepoOwnerAsync({ config, repository: repository as Repository });
return { owner, name: repository.name };
}
export async function archiveOnPushTarget({
config,
repository,
}: {
config: Partial<Config>;
repository: TargetRepositoryRow;
}): Promise<void> {
const target = createPushTargetFromConfig(config);
const { owner, name } = await resolvePushTargetLocation(config, repository);
await target.archiveRepository(owner, name);
}
export async function deleteOnPushTarget({
config,
repository,
}: {
config: Partial<Config>;
repository: TargetRepositoryRow;
}): Promise<void> {
const target = createPushTargetFromConfig(config);
const { owner, name } = await resolvePushTargetLocation(config, repository);
await target.deleteRepository(owner, name);
await removeCloneForRepository({ repository });
}
/** Drop the bare clone for a row. Safe to call when there is none. */
export async function removeCloneForRepository({
repository,
}: {
repository: CloneRepositoryFields;
}): Promise<void> {
await removeClone(clonePathForRepository(repository));
}
+390
View File
@@ -0,0 +1,390 @@
/**
* The push engine against real git on local file:// remotes: a bare
* "source", the engine's clone, and a bare "target" standing in for GitHub.
*/
import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, rm, stat, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import {
countChanges,
isHealthyBareClone,
isRetryableInBatches,
listRefs,
parsePushPorcelain,
removeClone,
runPushMirror,
type PushMirrorPlan,
} from "./engine";
import { GitCommandError, gitSubcommand, runGit } from "./git";
import { getPushLimiter, resetPushLimiter } from "./limiter";
import { CloneLockedError, lockPathFor } from "./lock";
import type {
EnsureRepositoryInput,
PushTarget,
PushTargetIdentity,
PushTargetRepository,
} from "./targets/types";
const GIT_IDENTITY = ["-c", "user.name=Push Engine Test", "-c", "user.email=push@example.com"];
async function exists(target: string): Promise<boolean> {
try {
await stat(target);
return true;
} catch {
return false;
}
}
/** A push target backed by bare repositories on disk. */
class LocalPushTarget implements PushTarget {
readonly kind = "local" as const;
readonly baseUrl: string;
readonly archived = new Set<string>();
readonly createdRepositories: string[] = [];
readonly defaultBranches = new Map<string, string>();
active = 0;
maxActive = 0;
constructor(
private readonly root: string,
private readonly ensureDelayMs = 0
) {
this.baseUrl = `file://${root}`;
}
private dirFor(owner: string, name: string): string {
return path.join(this.root, owner, `${name}.git`);
}
private describe(owner: string, name: string, created: boolean): PushTargetRepository {
const dir = this.dirFor(owner, name);
return {
owner,
name,
pushUrl: `file://${dir}`,
htmlUrl: `file://${dir}`,
isPrivate: false,
archived: this.archived.has(`${owner}/${name}`),
created,
};
}
pushCredentials() {
return null;
}
async testConnection(): Promise<PushTargetIdentity> {
return { login: "local" };
}
async getRepository(owner: string, name: string): Promise<PushTargetRepository | null> {
if (!(await exists(path.join(this.dirFor(owner, name), "HEAD")))) return null;
return this.describe(owner, name, false);
}
async ensureRepository(input: EnsureRepositoryInput): Promise<PushTargetRepository> {
this.active += 1;
this.maxActive = Math.max(this.maxActive, this.active);
try {
if (this.ensureDelayMs > 0) await Bun.sleep(this.ensureDelayMs);
const existing = await this.getRepository(input.owner, input.name);
if (existing) return existing;
const dir = this.dirFor(input.owner, input.name);
await mkdir(path.dirname(dir), { recursive: true });
await runGit(["init", "--bare", "--quiet", dir]);
this.createdRepositories.push(`${input.owner}/${input.name}`);
return this.describe(input.owner, input.name, true);
} finally {
this.active -= 1;
}
}
async setDefaultBranch(owner: string, name: string, branch: string): Promise<void> {
this.defaultBranches.set(`${owner}/${name}`, branch);
}
async archiveRepository(owner: string, name: string): Promise<void> {
this.archived.add(`${owner}/${name}`);
}
async deleteRepository(owner: string, name: string): Promise<void> {
await rm(this.dirFor(owner, name), { recursive: true, force: true });
}
}
interface SourceRepo {
bareDir: string;
workDir: string;
url: string;
}
async function commitFile(workDir: string, file: string, content: string, message: string): Promise<void> {
await writeFile(path.join(workDir, file), content);
await runGit(["-C", workDir, "add", "-A"]);
await runGit([...GIT_IDENTITY, "-C", workDir, "commit", "--quiet", "-m", message]);
}
async function createSourceRepo(root: string, name: string): Promise<SourceRepo> {
const bareDir = path.join(root, `${name}-source.git`);
const workDir = path.join(root, `${name}-work`);
await runGit(["init", "--bare", "--quiet", "--initial-branch=main", bareDir]);
await runGit(["init", "--quiet", "--initial-branch=main", workDir]);
await commitFile(workDir, "README.md", "hello\n", "initial commit");
await runGit(["-C", workDir, "checkout", "--quiet", "-b", "feature"]);
await commitFile(workDir, "feature.txt", "feature\n", "feature work");
await runGit(["-C", workDir, "checkout", "--quiet", "main"]);
await runGit([...GIT_IDENTITY, "-C", workDir, "tag", "-a", "v1.0.0", "-m", "first release"]);
await runGit(["-C", workDir, "remote", "add", "origin", bareDir]);
await runGit(["-C", workDir, "push", "--quiet", "origin", "main", "feature", "--tags"]);
return { bareDir, workDir, url: `file://${bareDir}` };
}
let root: string;
let target: LocalPushTarget;
function planFor(source: SourceRepo, name: string, overrides: Partial<PushMirrorPlan> = {}): PushMirrorPlan {
return {
clonePath: path.join(root, "mirrors", "example.test", "acme", `${name}.git`),
sourceCloneUrl: source.url,
sourceCredentials: null,
target,
owner: "acme",
name,
isPrivate: false,
description: "engine test",
defaultBranch: "main",
...overrides,
};
}
beforeAll(async () => {
root = await mkdtemp(path.join(os.tmpdir(), "push-engine-"));
});
afterAll(async () => {
await rm(root, { recursive: true, force: true });
});
beforeEach(() => {
target = new LocalPushTarget(path.join(root, "targets"));
resetPushLimiter(2);
});
describe("runPushMirror", () => {
test("first run clones the source, creates the target and pushes every branch and tag", async () => {
const source = await createSourceRepo(root, "first");
const outcome = await runPushMirror(planFor(source, "first"));
expect(outcome.cloned).toBe(true);
expect(outcome.pushed).toBe(true);
expect(outcome.batches).toBe(0);
expect(outcome.targetRepository.created).toBe(true);
expect(target.createdRepositories).toEqual(["acme/first"]);
const sourceRefs = await listRefs(source.bareDir);
const targetRefs = await listRefs(path.join(root, "targets", "acme", "first.git"));
expect([...targetRefs.entries()].sort()).toEqual([...sourceRefs.entries()].sort());
expect(targetRefs.has("refs/heads/main")).toBe(true);
expect(targetRefs.has("refs/heads/feature")).toBe(true);
expect(targetRefs.has("refs/tags/v1.0.0")).toBe(true);
expect(target.defaultBranches.get("acme/first")).toBe("main");
// No lock is left behind and the clone is a bare repository.
const clonePath = planFor(source, "first").clonePath;
expect(await exists(lockPathFor(clonePath))).toBe(false);
expect(await isHealthyBareClone(clonePath)).toBe(true);
});
test("a second run fetches the new commit and tag and pushes only the change; a third run is a no-op", async () => {
const source = await createSourceRepo(root, "incremental");
const plan = planFor(source, "incremental");
await runPushMirror(plan);
await commitFile(source.workDir, "CHANGELOG.md", "v2\n", "second commit");
await runGit([...GIT_IDENTITY, "-C", source.workDir, "tag", "v2.0.0"]);
await runGit(["-C", source.workDir, "push", "--quiet", "origin", "main", "--tags"]);
const second = await runPushMirror(plan);
expect(second.cloned).toBe(false);
expect(second.changedRefs).toBe(2);
expect(second.pushed).toBe(true);
expect(second.targetRepository.created).toBe(false);
const targetRefs = await listRefs(path.join(root, "targets", "acme", "incremental.git"));
const sourceRefs = await listRefs(source.bareDir);
expect(targetRefs.get("refs/heads/main")).toBe(sourceRefs.get("refs/heads/main"));
expect(targetRefs.has("refs/tags/v2.0.0")).toBe(true);
const third = await runPushMirror(plan);
expect(third.changedRefs).toBe(0);
expect(third.pushed).toBe(false);
});
test("a branch deleted on the source is pruned from the clone and the target", async () => {
const source = await createSourceRepo(root, "prune");
const plan = planFor(source, "prune");
await runPushMirror(plan);
await runGit(["-C", source.workDir, "push", "--quiet", "origin", "--delete", "feature"]);
const outcome = await runPushMirror(plan);
expect(outcome.changedRefs).toBe(1);
expect(outcome.refsAfter.has("refs/heads/feature")).toBe(false);
const targetRefs = await listRefs(path.join(root, "targets", "acme", "prune.git"));
expect(targetRefs.has("refs/heads/feature")).toBe(false);
expect(targetRefs.has("refs/heads/main")).toBe(true);
});
test("a force push on the source is replicated to the target", async () => {
const source = await createSourceRepo(root, "force");
const plan = planFor(source, "force");
await runPushMirror(plan);
const before = (await listRefs(source.bareDir)).get("refs/heads/main");
await runGit([...GIT_IDENTITY, "-C", source.workDir, "commit", "--quiet", "--amend", "-m", "rewritten history"]);
await runGit(["-C", source.workDir, "push", "--quiet", "--force", "origin", "main"]);
const after = (await listRefs(source.bareDir)).get("refs/heads/main");
expect(after).not.toBe(before);
const outcome = await runPushMirror(plan);
expect(outcome.pushed).toBe(true);
const targetRefs = await listRefs(path.join(root, "targets", "acme", "force.git"));
expect(targetRefs.get("refs/heads/main")).toBe(after);
});
test("pull request refs on the source never reach the clone or the target", async () => {
const source = await createSourceRepo(root, "hidden-refs");
const mainSha = (await listRefs(source.bareDir)).get("refs/heads/main")!;
await runGit(["-C", source.bareDir, "update-ref", "refs/pull/1/head", mainSha]);
const plan = planFor(source, "hidden-refs");
const outcome = await runPushMirror(plan);
const { stdout } = await runGit(["-C", plan.clonePath, "for-each-ref", "--format=%(refname)"]);
expect(stdout).not.toContain("refs/pull/");
expect([...outcome.refsAfter.keys()].some((ref) => ref.startsWith("refs/pull/"))).toBe(false);
const { stdout: targetOut } = await runGit([
"-C",
path.join(root, "targets", "acme", "hidden-refs.git"),
"for-each-ref",
"--format=%(refname)",
]);
expect(targetOut).not.toContain("refs/pull/");
});
test("a stale lock is taken over, a live lock is refused", async () => {
const source = await createSourceRepo(root, "locks");
const plan = planFor(source, "locks");
const lockPath = lockPathFor(plan.clonePath);
await mkdir(path.dirname(lockPath), { recursive: true });
const twoHoursAgo = new Date(Date.now() - 2 * 60 * 60 * 1000).toISOString();
await writeFile(lockPath, JSON.stringify({ pid: 1, takenAt: twoHoursAgo }));
const outcome = await runPushMirror(plan);
expect(outcome.cloned).toBe(true);
expect(await exists(lockPath)).toBe(false);
await writeFile(lockPath, JSON.stringify({ pid: 1, takenAt: new Date().toISOString() }));
await expect(runPushMirror(plan)).rejects.toBeInstanceOf(CloneLockedError);
await rm(lockPath, { force: true });
});
test("a half written clone directory is removed and cloned again", async () => {
const source = await createSourceRepo(root, "half");
const plan = planFor(source, "half");
await mkdir(plan.clonePath, { recursive: true });
await writeFile(path.join(plan.clonePath, "garbage"), "not a repository");
const outcome = await runPushMirror(plan);
expect(outcome.cloned).toBe(true);
expect(await exists(path.join(plan.clonePath, "garbage"))).toBe(false);
expect(await isHealthyBareClone(plan.clonePath)).toBe(true);
await removeClone(plan.clonePath);
expect(await exists(plan.clonePath)).toBe(false);
});
test("the global limit caps how many pushes run at once", async () => {
const slowTarget = new LocalPushTarget(path.join(root, "slow-targets"), 120);
target = slowTarget;
const a = await createSourceRepo(root, "limit-a");
const b = await createSourceRepo(root, "limit-b");
resetPushLimiter(1);
await Promise.all([runPushMirror(planFor(a, "limit-a")), runPushMirror(planFor(b, "limit-b"))]);
expect(slowTarget.maxActive).toBe(1);
expect(getPushLimiter().inFlight).toBe(0);
slowTarget.maxActive = 0;
resetPushLimiter(2);
await Promise.all([runPushMirror(planFor(a, "limit-a")), runPushMirror(planFor(b, "limit-b"))]);
expect(slowTarget.maxActive).toBe(2);
});
test("an archived target repository is refused before anything is pushed", async () => {
const source = await createSourceRepo(root, "archived");
const plan = planFor(source, "archived");
await runPushMirror(plan);
await target.archiveRepository("acme", "archived");
await expect(runPushMirror(plan)).rejects.toThrow(/archived/);
});
});
describe("helpers", () => {
test("parsePushPorcelain counts updates and up-to-date refs and lists rejections", () => {
const output = [
"To file:///tmp/target.git",
"=\trefs/heads/main:refs/heads/main\t[up to date]",
"*\trefs/tags/v2:refs/tags/v2\t[new tag]",
"+\trefs/heads/dev:refs/heads/dev\tabc...def (forced update)",
"-\t:refs/heads/old\t[deleted]",
"!\trefs/heads/locked:refs/heads/locked\t[remote rejected] (protected branch)",
"Done",
].join("\n");
const parsed = parsePushPorcelain(output);
expect(parsed.updated).toBe(3);
expect(parsed.upToDate).toBe(1);
expect(parsed.rejected).toEqual(["refs/heads/locked:refs/heads/locked"]);
});
test("countChanges counts added, moved and removed refs", () => {
const before = new Map([
["refs/heads/main", "a"],
["refs/heads/gone", "b"],
]);
const after = new Map([
["refs/heads/main", "c"],
["refs/tags/v1", "d"],
]);
expect(countChanges(before, after)).toBe(3);
expect(countChanges(after, after)).toBe(0);
});
test("gitSubcommand names the subcommand, not the -C path or -c values", () => {
expect(gitSubcommand(["-C", "/data/mirrors/x.git", "fetch", "--prune", "origin"])).toBe("fetch");
expect(gitSubcommand(["-c", "core.askPass=", "-C", "/tmp/x", "push", "--mirror"])).toBe("push");
expect(gitSubcommand(["clone", "--mirror", "https://example.com/a.git", "/tmp/a.git"])).toBe("clone");
expect(gitSubcommand(["--version"])).toBe("");
});
test("a failing git command reports its subcommand in the error message", async () => {
const dir = await mkdtemp(path.join(os.tmpdir(), "push-engine-nogit-"));
try {
await expect(runGit(["-C", dir, "fetch", "origin"])).rejects.toThrow(/^git fetch failed \(exit \d+\)/);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("isRetryableInBatches keeps credential and missing repository failures out of the batch retry", () => {
const auth = new GitCommandError("git push failed", ["push"], 128, "fatal: Authentication failed for 'https://x'");
const missing = new GitCommandError("git push failed", ["push"], 128, "remote: Repository not found.");
const size = new GitCommandError("git push failed", ["push"], 1, "error: RPC failed; HTTP 413 curl 22 The requested URL returned error: 413");
expect(isRetryableInBatches(auth)).toBe(false);
expect(isRetryableInBatches(missing)).toBe(false);
expect(isRetryableInBatches(size)).toBe(true);
expect(isRetryableInBatches(new Error("plain"))).toBe(false);
});
});
+268
View File
@@ -0,0 +1,268 @@
/**
* The push engine proper: keep a bare clone of the source up to date and
* push its branches and tags to the target.
*
* This module knows nothing about the database. It takes a plan (where the
* clone lives, how to reach the source, which target to push to) and
* returns what happened, so it can be exercised with local file:// remotes.
*
* Only refs/heads and refs/tags travel. Hosts refuse writes to their own
* namespaces (refs/pull on GitHub, refs/merge-requests on GitLab), so a
* literal `push --mirror` of a clone that carried them would fail; the
* explicit refspecs below give the same result for everything a host lets
* us write. Force and prune are on, so a rewritten or deleted branch on the
* source is rewritten or deleted on the target.
*/
import { mkdir, rm, stat } from "node:fs/promises";
import path from "node:path";
import { GitCommandError, parseRefList, runGit, type GitCredentials } from "./git";
import { getPushLimiter } from "./limiter";
import { lockPathFor, withCloneLock } from "./lock";
import type { PushTarget, PushTargetRepository } from "./targets/types";
export const MIRROR_REFSPECS = ["+refs/heads/*:refs/heads/*", "+refs/tags/*:refs/tags/*"] as const;
/** Refs per push when the single mirror push is refused for size. */
export const PUSH_BATCH_SIZE = 50;
export interface PushMirrorPlan {
clonePath: string;
sourceCloneUrl: string;
sourceCredentials: GitCredentials | null;
target: PushTarget;
owner: string;
name: string;
isPrivate: boolean;
description?: string | null;
defaultBranch?: string | null;
lockStaleAfterMs?: number;
log?: (message: string) => void;
}
export interface PushMirrorOutcome {
targetRepository: PushTargetRepository;
/** True when this run created the bare clone. */
cloned: boolean;
refsBefore: Map<string, string>;
refsAfter: Map<string, string>;
/** Refs the fetch added, moved or removed in the clone. */
changedRefs: number;
/** False when the target already had everything. */
pushed: boolean;
/** Number of batched pushes used; 0 when the single push went through. */
batches: number;
}
export async function runPushMirror(plan: PushMirrorPlan): Promise<PushMirrorOutcome> {
return withCloneLock(plan.clonePath, () => getPushLimiter().run(() => execute(plan)), {
staleAfterMs: plan.lockStaleAfterMs,
});
}
async function execute(plan: PushMirrorPlan): Promise<PushMirrorOutcome> {
const log = plan.log ?? (() => {});
const { cloned, refsBefore } = await ensureClone(plan, log);
const refsAfter = await listRefs(plan.clonePath);
const changedRefs = countChanges(refsBefore, refsAfter);
log(
cloned
? `cloned ${refsAfter.size} refs from the source`
: `fetched from the source: ${changedRefs} ref(s) changed, ${refsAfter.size} total`
);
const targetRepository = await plan.target.ensureRepository({
owner: plan.owner,
name: plan.name,
isPrivate: plan.isPrivate,
description: plan.description,
});
if (targetRepository.archived) {
throw new Error(
`${targetRepository.owner}/${targetRepository.name} is archived on ${plan.target.baseUrl}. Unarchive it there before mirroring into it.`
);
}
if (targetRepository.created) {
log(`created ${targetRepository.owner}/${targetRepository.name} on ${plan.target.baseUrl}`);
}
const { pushed, batches } = await pushAll(plan, targetRepository.pushUrl, refsAfter, log);
log(pushed ? `pushed to ${targetRepository.htmlUrl}` : `${targetRepository.htmlUrl} was already up to date`);
const wanted = plan.defaultBranch?.trim();
if (
wanted &&
refsAfter.has(`refs/heads/${wanted}`) &&
(targetRepository.created || pushed) &&
plan.target.setDefaultBranch
) {
try {
await plan.target.setDefaultBranch(targetRepository.owner, targetRepository.name, wanted);
} catch (error) {
log(`could not set the default branch to ${wanted}: ${error instanceof Error ? error.message : String(error)}`);
}
}
return { targetRepository, cloned, refsBefore, refsAfter, changedRefs, pushed, batches };
}
async function pathExists(target: string): Promise<boolean> {
try {
await stat(target);
return true;
} catch {
return false;
}
}
/** A usable bare clone has a HEAD and git agrees it is bare. */
export async function isHealthyBareClone(clonePath: string): Promise<boolean> {
if (!(await pathExists(path.join(clonePath, "HEAD")))) return false;
try {
const { stdout } = await runGit(["-C", clonePath, "rev-parse", "--is-bare-repository"]);
return stdout.trim() === "true";
} catch {
return false;
}
}
async function configureClone(plan: PushMirrorPlan): Promise<void> {
const { clonePath, sourceCloneUrl } = plan;
await runGit(["-C", clonePath, "remote", "set-url", "origin", sourceCloneUrl]);
// Exactly two fetch refspecs, whatever an earlier run left behind.
await runGit(["-C", clonePath, "config", "--replace-all", "remote.origin.fetch", MIRROR_REFSPECS[0]]);
await runGit(["-C", clonePath, "config", "--add", "remote.origin.fetch", MIRROR_REFSPECS[1]]);
}
async function ensureClone(
plan: PushMirrorPlan,
log: (message: string) => void
): Promise<{ cloned: boolean; refsBefore: Map<string, string> }> {
const { clonePath, sourceCloneUrl, sourceCredentials } = plan;
if (!(await isHealthyBareClone(clonePath))) {
if (await pathExists(clonePath)) {
log(`removing a half written clone at ${clonePath}`);
await rm(clonePath, { recursive: true, force: true });
}
await mkdir(path.dirname(clonePath), { recursive: true });
await runGit(["clone", "--bare", "--quiet", "--no-tags", sourceCloneUrl, clonePath], {
credentials: sourceCredentials,
});
await configureClone(plan);
// A bare clone already carries the branches; the configured refspecs
// bring the tags and prune anything the source no longer has.
await runGit(["-C", clonePath, "fetch", "--quiet", "--prune", "--prune-tags", "--no-recurse-submodules", "origin"], {
credentials: sourceCredentials,
});
return { cloned: true, refsBefore: new Map() };
}
await configureClone(plan);
const refsBefore = await listRefs(clonePath);
await runGit(["-C", clonePath, "fetch", "--quiet", "--prune", "--prune-tags", "--no-recurse-submodules", "origin"], {
credentials: sourceCredentials,
});
return { cloned: false, refsBefore };
}
export async function listRefs(clonePath: string): Promise<Map<string, string>> {
const { stdout } = await runGit([
"-C",
clonePath,
"for-each-ref",
"--format=%(objectname) %(refname)",
"refs/heads",
"refs/tags",
]);
return parseRefList(stdout);
}
export function countChanges(before: Map<string, string>, after: Map<string, string>): number {
let changed = 0;
for (const [ref, sha] of after) {
if (before.get(ref) !== sha) changed += 1;
}
for (const ref of before.keys()) {
if (!after.has(ref)) changed += 1;
}
return changed;
}
/** Porcelain push output: one line per ref, first character is the status flag. */
export function parsePushPorcelain(output: string): { updated: number; upToDate: number; rejected: string[] } {
let updated = 0;
let upToDate = 0;
const rejected: string[] = [];
for (const line of output.split("\n")) {
if (!line || line.startsWith("To ") || line.startsWith("Done")) continue;
const flag = line[0];
const refs = line.slice(1).split("\t")[1] ?? "";
if (flag === "=") upToDate += 1;
else if (flag === "!") rejected.push(refs || line.trim());
else if (" +-*".includes(flag)) updated += 1;
}
return { updated, upToDate, rejected };
}
const NON_RETRYABLE_PUSH = /authentication failed|could not read username|permission|denied|not found|repository .* does not exist|403|401|archived|read-only/i;
/** Failures that batching cannot help with: bad credentials, missing repository. */
export function isRetryableInBatches(error: unknown): boolean {
if (!(error instanceof GitCommandError)) return false;
return !NON_RETRYABLE_PUSH.test(`${error.message}\n${error.stderr}`);
}
async function pushAll(
plan: PushMirrorPlan,
pushUrl: string,
refs: Map<string, string>,
log: (message: string) => void
): Promise<{ pushed: boolean; batches: number }> {
const credentials = plan.target.pushCredentials();
const { clonePath } = plan;
try {
const { stdout } = await runGit(
["-C", clonePath, "push", "--no-progress", "--prune", "--porcelain", pushUrl, ...MIRROR_REFSPECS],
{ credentials }
);
const parsed = parsePushPorcelain(stdout);
return { pushed: parsed.updated > 0, batches: 0 };
} catch (error) {
if (!isRetryableInBatches(error)) throw error;
const reason = error instanceof Error ? error.message : String(error);
log(`single push was refused (${reason}); retrying in batches of ${PUSH_BATCH_SIZE} refs`);
}
const heads = [...refs.keys()].filter((ref) => ref.startsWith("refs/heads/"));
const tags = [...refs.keys()].filter((ref) => ref.startsWith("refs/tags/"));
let batches = 0;
for (const group of [heads, tags]) {
for (let i = 0; i < group.length; i += PUSH_BATCH_SIZE) {
const chunk = group.slice(i, i + PUSH_BATCH_SIZE).map((ref) => `+${ref}:${ref}`);
batches += 1;
try {
await runGit(["-C", clonePath, "push", "--no-progress", "--porcelain", pushUrl, ...chunk], { credentials });
} catch (error) {
throw new Error(
`Pushing to ${pushUrl.replace(/\/\/[^@]*@/, "//")} failed even in batches of ${PUSH_BATCH_SIZE} refs (batch ${batches}): ${
error instanceof Error ? error.message : String(error)
}. The target may limit the size of a single push; try again later or reduce the repository first.`
);
}
}
}
// The batches only added and updated refs; a final prune removes what the source dropped.
batches += 1;
await runGit(["-C", clonePath, "push", "--no-progress", "--prune", "--porcelain", pushUrl, ...MIRROR_REFSPECS], {
credentials,
});
return { pushed: true, batches };
}
/** Remove a repository's bare clone and its lock file. */
export async function removeClone(clonePath: string): Promise<void> {
await rm(clonePath, { recursive: true, force: true });
await rm(lockPathFor(clonePath), { force: true });
}
+173
View File
@@ -0,0 +1,173 @@
/**
* Running git for the push engine.
*
* Credentials never touch a URL or the disk. Each invocation gets an inline
* credential helper that answers from two environment variables set on the
* child process only, so the remote URL git stores in the bare clone stays
* clean and the token cannot leak into `git remote -v`, logs or a core dump
* of the repository directory.
*/
export interface GitCredentials {
username: string;
token: string;
}
export interface RunGitOptions {
cwd?: string;
/** Credentials for the http(s) remote this command talks to, if any. */
credentials?: GitCredentials | null;
/** Kill the process after this long. Defaults to one hour: first clones can be big. */
timeoutMs?: number;
}
export interface GitResult {
stdout: string;
stderr: string;
}
export class GitCommandError extends Error {
constructor(
message: string,
public readonly args: string[],
public readonly exitCode: number | null,
public readonly stderr: string
) {
super(message);
this.name = "GitCommandError";
}
}
const DEFAULT_TIMEOUT_MS = 60 * 60 * 1000;
const CREDENTIAL_USER_ENV = "GITEA_MIRROR_GIT_USERNAME";
const CREDENTIAL_TOKEN_ENV = "GITEA_MIRROR_GIT_TOKEN";
/**
* The helper git runs to obtain credentials. A helper starting with "!" is
* executed through the shell; it prints the answer on stdout.
*/
const INLINE_CREDENTIAL_HELPER =
`!f() { printf 'username=%s\\npassword=%s\\n' "$${CREDENTIAL_USER_ENV}" "$${CREDENTIAL_TOKEN_ENV}"; }; f`;
export function maskSecret(text: string, secret?: string | null): string {
if (!secret) return text;
return text.split(secret).join("***");
}
/** Arguments that go before the git subcommand. */
export function credentialArgs(credentials?: GitCredentials | null): string[] {
const args = [
// Never fall back to a terminal or GUI prompt: fail instead of hanging.
"-c",
"core.askPass=",
];
if (credentials) {
// An empty value resets the helper list, so a global helper (keychain,
// store) cannot answer with somebody else's token first.
args.push("-c", "credential.helper=", "-c", `credential.helper=${INLINE_CREDENTIAL_HELPER}`);
}
return args;
}
export async function runGit(args: string[], options: RunGitOptions = {}): Promise<GitResult> {
const { cwd, credentials = null, timeoutMs = DEFAULT_TIMEOUT_MS } = options;
const env: Record<string, string | undefined> = {
...process.env,
GIT_TERMINAL_PROMPT: "0",
LC_ALL: "C",
};
if (credentials) {
env[CREDENTIAL_USER_ENV] = credentials.username;
env[CREDENTIAL_TOKEN_ENV] = credentials.token;
} else {
delete env[CREDENTIAL_USER_ENV];
delete env[CREDENTIAL_TOKEN_ENV];
}
const fullArgs = [...credentialArgs(credentials), ...args];
// The kill timer lives in the runtime, not in a JS timer: the test setup
// replaces setTimeout with an immediate call, which would kill every git
// process the moment it starts.
const proc = Bun.spawn({
cmd: ["git", ...fullArgs],
cwd,
env,
stdin: "ignore",
stdout: "pipe",
stderr: "pipe",
timeout: timeoutMs,
killSignal: "SIGTERM",
});
const [stdout, stderr, exitCode] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
const token = credentials?.token;
const safeStdout = maskSecret(stdout, token);
const safeStderr = maskSecret(stderr, token);
const subcommand = gitSubcommand(args);
if (proc.signalCode) {
throw new GitCommandError(
`git ${subcommand} was stopped (${proc.signalCode}) after ${Math.round(timeoutMs / 1000)}s`,
args,
exitCode,
safeStderr
);
}
if (exitCode !== 0) {
const detail = [safeStderr, safeStdout].filter(Boolean).join("\n").trim();
throw new GitCommandError(
`git ${subcommand} failed (exit ${exitCode}): ${summarize(detail) || "unknown git error"}`,
args,
exitCode,
safeStderr
);
}
return { stdout: safeStdout, stderr: safeStderr };
}
/**
* The git subcommand in an argv, for error messages. Skips flags and the
* value that follows `-C` or `-c`, so `["-C", dir, "fetch"]` yields "fetch".
*/
export function gitSubcommand(args: string[]): string {
let skipNext = false;
for (const arg of args) {
if (skipNext) {
skipNext = false;
continue;
}
if (arg === "-C" || arg === "-c") {
skipNext = true;
continue;
}
if (!arg.startsWith("-")) return arg;
}
return "";
}
/** Keep the interesting last lines of git's output for an error message. */
function summarize(detail: string): string {
const lines = detail
.split("\n")
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("remote: Resolving deltas"));
return lines.slice(-4).join(" | ").slice(0, 600);
}
/** Parse `git for-each-ref` output into a ref -> sha map. */
export function parseRefList(output: string): Map<string, string> {
const refs = new Map<string, string>();
for (const line of output.split("\n")) {
const trimmed = line.trim();
if (!trimmed) continue;
const [sha, ref] = trimmed.split(/\s+/, 2);
if (sha && ref) refs.set(ref, sha);
}
return refs;
}
+11
View File
@@ -0,0 +1,11 @@
/**
* Push engine: mirrors to hosts without a pull mirror API (GitHub, GitLab)
* by keeping a bare clone and pushing it. See docs/PUSH_TARGETS.md.
*/
export * from "./engine";
export * from "./git";
export * from "./limiter";
export * from "./lock";
export * from "./mirror";
export * from "./paths";
export * from "./targets";
+69
View File
@@ -0,0 +1,69 @@
/**
* A small global limit on concurrent git runs. API calls are cheap; a
* clone or a mirror push is not, so the engine keeps only a couple in
* flight no matter how many mirror jobs the routes start at once.
*/
export const DEFAULT_PUSH_CONCURRENCY = 2;
export function resolvePushConcurrency(env: NodeJS.ProcessEnv = process.env): number {
const raw = env.PUSH_CONCURRENCY?.trim();
if (!raw) return DEFAULT_PUSH_CONCURRENCY;
const parsed = Number.parseInt(raw, 10);
if (!Number.isFinite(parsed) || parsed < 1) return DEFAULT_PUSH_CONCURRENCY;
return Math.min(parsed, 32);
}
export class Semaphore {
private active = 0;
private readonly waiting: Array<() => void> = [];
constructor(public readonly limit: number) {}
get inFlight(): number {
return this.active;
}
get queued(): number {
return this.waiting.length;
}
async acquire(): Promise<() => void> {
if (this.active < this.limit) {
this.active += 1;
return () => this.release();
}
await new Promise<void>((resolve) => this.waiting.push(resolve));
this.active += 1;
return () => this.release();
}
private release(): void {
this.active -= 1;
const next = this.waiting.shift();
if (next) next();
}
async run<T>(fn: () => Promise<T>): Promise<T> {
const release = await this.acquire();
try {
return await fn();
} finally {
release();
}
}
}
let shared: Semaphore | null = null;
/** The process-wide limiter, sized from PUSH_CONCURRENCY on first use. */
export function getPushLimiter(): Semaphore {
if (!shared) shared = new Semaphore(resolvePushConcurrency());
return shared;
}
/** Test hook: replace the shared limiter. */
export function resetPushLimiter(limit?: number): Semaphore {
shared = new Semaphore(limit ?? resolvePushConcurrency());
return shared;
}
+92
View File
@@ -0,0 +1,92 @@
/**
* One git process per repository at a time.
*
* The lock is a file next to the clone directory (the directory itself may
* not exist yet on the first run). It records the pid and the time it was
* taken; a lock older than `staleAfterMs` belongs to a run that died and
* is taken over.
*/
import { open, readFile, rm, stat, mkdir } from "node:fs/promises";
import path from "node:path";
export const DEFAULT_LOCK_STALE_MS = 60 * 60 * 1000;
export class CloneLockedError extends Error {
constructor(public readonly lockPath: string, public readonly heldSince: Date | null) {
super(
`Another mirror run holds the lock for this repository${
heldSince ? ` since ${heldSince.toISOString()}` : ""
}. Try again once it finishes.`
);
this.name = "CloneLockedError";
}
}
export function lockPathFor(clonePath: string): string {
return `${clonePath.replace(/[\\/]+$/, "")}.lock`;
}
async function tryTakeLock(lockPath: string): Promise<boolean> {
try {
const handle = await open(lockPath, "wx");
try {
await handle.writeFile(JSON.stringify({ pid: process.pid, takenAt: new Date().toISOString() }));
} finally {
await handle.close();
}
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "EEXIST") return false;
throw error;
}
}
async function lockAge(lockPath: string): Promise<{ ageMs: number; takenAt: Date } | null> {
try {
const info = await stat(lockPath);
let takenAt = info.mtime;
try {
const parsed = JSON.parse(await readFile(lockPath, "utf8")) as { takenAt?: string };
if (parsed.takenAt) {
const fromFile = new Date(parsed.takenAt);
if (!Number.isNaN(fromFile.getTime())) takenAt = fromFile;
}
} catch {
// Unreadable content: the mtime is good enough.
}
return { ageMs: Date.now() - takenAt.getTime(), takenAt };
} catch {
return null;
}
}
/**
* Run `fn` while holding the clone lock. A stale lock is removed and taken
* over; a live one raises CloneLockedError without waiting.
*/
export async function withCloneLock<T>(
clonePath: string,
fn: () => Promise<T>,
{ staleAfterMs = DEFAULT_LOCK_STALE_MS }: { staleAfterMs?: number } = {}
): Promise<T> {
const lockPath = lockPathFor(clonePath);
await mkdir(path.dirname(lockPath), { recursive: true });
let taken = await tryTakeLock(lockPath);
if (!taken) {
const age = await lockAge(lockPath);
if (age && age.ageMs > staleAfterMs) {
await rm(lockPath, { force: true });
taken = await tryTakeLock(lockPath);
}
if (!taken) {
throw new CloneLockedError(lockPath, age?.takenAt ?? null);
}
}
try {
return await fn();
} finally {
await rm(lockPath, { force: true });
}
}
+210
View File
@@ -0,0 +1,210 @@
/**
* The database aware side of the push engine.
*
* Mirrors the statuses, mirror jobs and activity rows the Gitea path
* writes, so the dashboard, recovery and cleanup see a push target exactly
* like a pull mirror. The git work itself lives in engine.ts.
*/
import { eq } from "drizzle-orm";
import { db, repositories } from "@/lib/db";
import type { Config, Repository } from "@/lib/db/schema";
import { createMirrorJob } from "@/lib/helpers";
import { getGiteaRepoOwnerAsync } from "@/lib/gitea";
import {
assertRepositoryMatchesConfiguredSource,
resolveSourceConnection,
} from "@/lib/source-providers";
import { getDecryptedGitHubToken } from "@/lib/utils/config-encryption";
import { buildSourceAuthPayload } from "@/lib/utils/mirror-source-auth";
import {
assertRepositoryMatchesConfiguredDestination,
createPushTargetFromConfig,
resolveDestinationIdentity,
resolvePushDestinationKind,
} from "@/lib/destination-connection";
import { DESTINATION_PROVIDER_LABELS } from "@/lib/destination-kinds";
import { repoStatusEnum } from "@/types/Repository";
import { runPushMirror, type PushMirrorOutcome } from "./engine";
import type { GitCredentials } from "./git";
import { clonePathForRepository } from "./paths";
import type { PushTarget } from "./targets/types";
export type PushMirrorMode = "mirror" | "sync";
export interface PushMirrorRepositoryOptions {
config: Partial<Config>;
repository: Repository;
/** "mirror" for the first push, "sync" for later ones. Picks statuses and the job type. */
mode?: PushMirrorMode;
/** Test hook: use this target instead of the one built from the config. */
target?: PushTarget;
}
/** Split a recorded `owner/name` mirror location. */
export function parseMirroredLocation(location: string | null | undefined): { owner: string; name: string } | null {
const trimmed = (location || "").trim();
const slash = trimmed.lastIndexOf("/");
if (slash <= 0 || slash === trimmed.length - 1) return null;
return { owner: trimmed.slice(0, slash), name: trimmed.slice(slash + 1) };
}
/** The credentials git uses to fetch from the source, per source host. */
export function resolveSourceCredentials(config: Partial<Config>, repository: Repository): GitCredentials | null {
const token = config.githubConfig?.token ? getDecryptedGitHubToken(config as Config) : "";
const connection = resolveSourceConnection(config, { token });
const payload = buildSourceAuthPayload({
provider: connection.provider,
token,
username: connection.username,
repositoryOwner: repository.owner,
});
if (!("auth_username" in payload) || !payload.auth_password) return null;
return { username: payload.auth_username, token: payload.auth_password };
}
/** Private on the target unless the source repository is public and visibility is preserved. */
export function resolveTargetPrivacy(config: Partial<Config>, repository: Repository): boolean {
if (repository.isPrivate || repository.visibility === "private" || repository.visibility === "internal") {
return true;
}
const preserve = config.giteaConfig?.preserveVisibility ?? true;
if (preserve) return false;
return config.giteaConfig?.visibility === "private";
}
export async function pushMirrorRepository({
config,
repository,
mode,
target,
}: PushMirrorRepositoryOptions): Promise<PushMirrorOutcome> {
if (!config.userId || !config.giteaConfig?.url || !config.giteaConfig?.token) {
throw new Error("A destination URL and token are required.");
}
if (!config.giteaConfig.defaultOwner) {
throw new Error("The destination account name is required.");
}
const kind = resolvePushDestinationKind(config);
const label = DESTINATION_PROVIDER_LABELS[kind];
const destination = resolveDestinationIdentity(config);
// Never send another host's token in either direction.
assertRepositoryMatchesConfiguredSource({ repository, config });
assertRepositoryMatchesConfiguredDestination({ repository, config });
const recorded = parseMirroredLocation(repository.mirroredLocation);
const effectiveMode: PushMirrorMode = mode ?? (recorded ? "sync" : "mirror");
const owner = recorded?.owner ?? (await getGiteaRepoOwnerAsync({ config, repository }));
const name = recorded?.name ?? repository.name;
const location = `${owner}/${name}`;
const busyStatus = effectiveMode === "mirror" ? "mirroring" : "syncing";
const doneStatus = effectiveMode === "mirror" ? "mirrored" : "synced";
const jobType = effectiveMode === "mirror" ? "mirror" : "sync";
const logPrefix = `[Push ${label}] ${repository.fullName}:`;
await db
.update(repositories)
.set({
status: repoStatusEnum.parse(busyStatus),
mirroredLocation: location,
destinationProvider: destination.provider,
destinationUrl: destination.url,
updatedAt: new Date(),
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message:
effectiveMode === "mirror"
? `Started mirroring repository: ${repository.name}`
: `Started syncing repository: ${repository.name}`,
details: `Repository ${repository.fullName} is being pushed to ${label} at ${location}.`,
status: busyStatus,
jobType,
});
try {
const outcome = await runPushMirror({
clonePath: clonePathForRepository(repository),
sourceCloneUrl: repository.cloneUrl,
sourceCredentials: resolveSourceCredentials(config, repository),
target: target ?? createPushTargetFromConfig(config),
owner,
name,
isPrivate: resolveTargetPrivacy(config, repository),
description: repository.description,
defaultBranch: repository.defaultBranch,
log: (message) => console.log(`${logPrefix} ${message}`),
});
const finalLocation = `${outcome.targetRepository.owner}/${outcome.targetRepository.name}`;
await db
.update(repositories)
.set({
status: repoStatusEnum.parse(doneStatus),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: finalLocation,
destinationProvider: destination.provider,
destinationUrl: destination.url,
})
.where(eq(repositories.id, repository.id!));
const summary =
outcome.cloned || outcome.targetRepository.created
? `first push of ${outcome.refsAfter.size} refs`
: outcome.pushed
? `${outcome.changedRefs} ref(s) updated`
: "already up to date";
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message:
effectiveMode === "mirror"
? `Successfully mirrored repository: ${repository.name}`
: `Successfully synced repository: ${repository.name}`,
details: `Repository ${repository.fullName} was pushed to ${label} at ${finalLocation} (${summary}${
outcome.batches ? `, in ${outcome.batches} batches` : ""
}).`,
status: doneStatus,
jobType,
});
return outcome;
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
console.error(`${logPrefix} failed: ${message}`);
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("failed"),
updatedAt: new Date(),
errorMessage: message,
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message:
effectiveMode === "mirror"
? `Failed to mirror repository: ${repository.name}`
: `Failed to sync repository: ${repository.name}`,
details: `Repository ${repository.fullName} could not be pushed to ${label}. Error: ${message}`,
status: "failed",
jobType,
});
throw new Error(
effectiveMode === "mirror" ? `Failed to mirror repository: ${message}` : `Failed to sync repository: ${message}`
);
}
}
+66
View File
@@ -0,0 +1,66 @@
/**
* Where the push engine keeps its bare clones.
*
* One clone per repository under `<data dir>/mirrors/<source host>/<owner>/<name>.git`,
* next to the SQLite file so a backup of the data directory carries the
* clones too. `MIRROR_CLONE_DIR` moves the whole tree elsewhere.
*/
import path from "node:path";
import { getRepositorySource, type RepositorySourceFields } from "@/lib/source-providers/kinds";
/** The directory the SQLite database lives in, derived the same way src/lib/db does. */
export function resolveDataDir(): string {
const url = process.env.DATABASE_URL?.trim();
if (!url) {
return path.join(process.cwd(), "data");
}
const raw = url.startsWith("sqlite://")
? url.slice("sqlite://".length)
: url.startsWith("file:")
? url.slice("file:".length)
: url;
return path.dirname(path.resolve(raw));
}
export function resolveMirrorCloneRoot(): string {
const configured = process.env.MIRROR_CLONE_DIR?.trim();
return path.resolve(configured || path.join(resolveDataDir(), "mirrors"));
}
/** Keep directory names to a safe character set; anything else becomes "_". */
export function sanitizePathSegment(input: string): string {
const cleaned = input.replace(/[^a-zA-Z0-9._-]/g, "_").replace(/^\.+/, "_");
return cleaned || "_";
}
function hostOf(url: string): string {
try {
return new URL(url).host.toLowerCase();
} catch {
return "unknown-host";
}
}
export interface ClonePathInput {
/** Base URL of the source host, e.g. https://github.com. */
sourceUrl: string;
owner: string;
name: string;
}
export type CloneRepositoryFields = RepositorySourceFields & { owner: string; name: string };
/** The clone path for a stored repository row: its source host, owner and name. */
export function clonePathForRepository(repo: CloneRepositoryFields, root?: string): string {
return clonePathFor({ sourceUrl: getRepositorySource(repo).url, owner: repo.owner, name: repo.name }, root);
}
/** Absolute path of the bare clone for a repository. */
export function clonePathFor(input: ClonePathInput, root: string = resolveMirrorCloneRoot()): string {
return path.join(
root,
sanitizePathSegment(hostOf(input.sourceUrl)),
sanitizePathSegment(input.owner),
`${sanitizePathSegment(input.name.replace(/\.git$/i, ""))}.git`
);
}
+189
View File
@@ -0,0 +1,189 @@
/**
* GitHub as a push target.
*
* Talks to the REST API with the account's token. github.com uses
* api.github.com; any other host is treated as GitHub Enterprise and gets
* the `/api/v3` prefix on the instance URL.
*/
import type { GitCredentials } from "../git";
import { isTargetNotFound, targetFetch } from "./http";
import {
PushTargetError,
type EnsureRepositoryInput,
type PushTarget,
type PushTargetConnection,
type PushTargetIdentity,
type PushTargetRepository,
} from "./types";
interface GitHubRepo {
name: string;
full_name?: string;
owner?: { login?: string };
private?: boolean;
archived?: boolean;
clone_url?: string;
html_url?: string;
default_branch?: string;
}
const MAX_DESCRIPTION = 350;
export function githubApiRootFor(baseUrl: string): string {
let host = "";
try {
host = new URL(baseUrl).host.toLowerCase();
} catch {
host = "";
}
if (!host || host === "github.com" || host === "www.github.com") {
return "https://api.github.com";
}
return `${baseUrl.replace(/\/+$/, "")}/api/v3`;
}
export class GitHubPushTarget implements PushTarget {
readonly kind = "github" as const;
readonly baseUrl: string;
private readonly apiRoot: string;
constructor(private readonly connection: PushTargetConnection) {
this.baseUrl = connection.url.replace(/\/+$/, "");
this.apiRoot = githubApiRootFor(this.baseUrl);
}
private headers(): Record<string, string> {
return {
Authorization: `Bearer ${this.connection.token}`,
Accept: "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "gitea-mirror",
};
}
pushCredentials(): GitCredentials {
// GitHub accepts any user name with a token as the password; the account
// name keeps audit logs readable.
return { username: this.connection.username.trim() || "x-access-token", token: this.connection.token };
}
async testConnection(): Promise<PushTargetIdentity> {
const { data } = await targetFetch<{ login?: string }>(`${this.apiRoot}/user`, {
headers: this.headers(),
});
if (!data?.login) {
throw new PushTargetError("GitHub did not return a user for this token.");
}
const label = this.apiRoot === "https://api.github.com" ? "GitHub" : "GitHub Enterprise";
return { login: data.login, label };
}
private toRepository(repo: GitHubRepo, created: boolean): PushTargetRepository {
const owner = repo.owner?.login || repo.full_name?.split("/")[0] || "";
return {
owner,
name: repo.name,
pushUrl: repo.clone_url || `${this.baseUrl}/${owner}/${repo.name}.git`,
htmlUrl: repo.html_url || `${this.baseUrl}/${owner}/${repo.name}`,
isPrivate: !!repo.private,
archived: !!repo.archived,
created,
};
}
async getRepository(owner: string, name: string): Promise<PushTargetRepository | null> {
try {
const { data } = await targetFetch<GitHubRepo>(
`${this.apiRoot}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}`,
{ headers: this.headers() }
);
return this.toRepository(data, false);
} catch (error) {
if (isTargetNotFound(error)) return null;
throw error;
}
}
private ownsNamespace(owner: string): boolean {
return owner.trim().toLowerCase() === this.connection.username.trim().toLowerCase();
}
async ensureRepository(input: EnsureRepositoryInput): Promise<PushTargetRepository> {
const existing = await this.getRepository(input.owner, input.name);
if (existing) return existing;
const body = {
name: input.name,
private: input.isPrivate,
description: (input.description || "").slice(0, MAX_DESCRIPTION) || undefined,
auto_init: false,
has_issues: false,
has_projects: false,
has_wiki: false,
};
const url = this.ownsNamespace(input.owner)
? `${this.apiRoot}/user/repos`
: `${this.apiRoot}/orgs/${encodeURIComponent(input.owner)}/repos`;
try {
const { data } = await targetFetch<GitHubRepo>(url, {
method: "POST",
headers: this.headers(),
body,
});
return this.toRepository(data, true);
} catch (error) {
if (error instanceof PushTargetError) {
if (error.status === 404 && !this.ownsNamespace(input.owner)) {
throw new PushTargetError(
`GitHub organization "${input.owner}" was not found, or the token cannot create repositories in it. ` +
"GitHub organizations cannot be created through the API: create it first and make the token's user a member with repository creation rights.",
error.status,
error.url
);
}
if (error.status === 422) {
// Lost a race with another run, or the name exists in another casing.
const again = await this.getRepository(input.owner, input.name);
if (again) return again;
}
}
throw error;
}
}
async setDefaultBranch(owner: string, name: string, branch: string): Promise<void> {
await targetFetch(`${this.apiRoot}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}`, {
method: "PATCH",
headers: this.headers(),
body: { default_branch: branch },
});
}
async archiveRepository(owner: string, name: string): Promise<void> {
await targetFetch(`${this.apiRoot}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}`, {
method: "PATCH",
headers: this.headers(),
body: { archived: true },
});
}
async deleteRepository(owner: string, name: string): Promise<void> {
try {
await targetFetch(`${this.apiRoot}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}`, {
method: "DELETE",
headers: this.headers(),
});
} catch (error) {
if (error instanceof PushTargetError && error.status === 403) {
throw new PushTargetError(
`GitHub refused to delete ${owner}/${name}: the token needs the delete_repo scope.`,
error.status,
error.url
);
}
if (isTargetNotFound(error)) return;
throw error;
}
}
}
+224
View File
@@ -0,0 +1,224 @@
/**
* GitLab as a push target.
*
* Talks to the v4 REST API with a personal access token. Projects live in
* a namespace: the token's own user, or a group looked up by path (and
* created as a top level group when missing).
*/
import type { GitCredentials } from "../git";
import { isTargetNotFound, targetFetch } from "./http";
import {
PushTargetError,
type EnsureRepositoryInput,
type PushTarget,
type PushTargetConnection,
type PushTargetIdentity,
type PushTargetRepository,
} from "./types";
interface GitLabProject {
id: number;
name: string;
path: string;
path_with_namespace?: string;
namespace?: { full_path?: string; path?: string };
visibility?: string;
archived?: boolean;
http_url_to_repo?: string;
web_url?: string;
default_branch?: string;
/** Set while a project waits in delayed deletion; the old path still redirects to it. */
marked_for_deletion_at?: string | null;
marked_for_deletion_on?: string | null;
}
interface GitLabNamespace {
id: number;
full_path?: string;
path?: string;
kind?: string;
}
const MAX_DESCRIPTION = 2000;
export class GitLabPushTarget implements PushTarget {
readonly kind = "gitlab" as const;
readonly baseUrl: string;
private readonly apiRoot: string;
constructor(private readonly connection: PushTargetConnection) {
this.baseUrl = connection.url.replace(/\/+$/, "");
this.apiRoot = `${this.baseUrl}/api/v4`;
}
private headers(): Record<string, string> {
return { "PRIVATE-TOKEN": this.connection.token, "User-Agent": "gitea-mirror" };
}
pushCredentials(): GitCredentials {
// GitLab documents "oauth2" as the user name for token pushes over HTTPS.
return { username: "oauth2", token: this.connection.token };
}
async testConnection(): Promise<PushTargetIdentity> {
const { data } = await targetFetch<{ username?: string }>(`${this.apiRoot}/user`, {
headers: this.headers(),
});
if (!data?.username) {
throw new PushTargetError("GitLab did not return a user for this token.");
}
let label = "GitLab";
try {
const version = await targetFetch<{ version?: string }>(`${this.apiRoot}/version`, {
headers: this.headers(),
});
if (version.data?.version) label = `GitLab ${version.data.version}`;
} catch {
// The version is decoration only.
}
return { login: data.username, label };
}
private projectPath(owner: string, name: string): string {
return encodeURIComponent(`${owner.replace(/^\/+|\/+$/g, "")}/${name}`);
}
private toRepository(project: GitLabProject, created: boolean): PushTargetRepository {
const owner =
project.namespace?.full_path ||
project.path_with_namespace?.split("/").slice(0, -1).join("/") ||
"";
return {
owner,
name: project.path || project.name,
pushUrl: project.http_url_to_repo || `${this.baseUrl}/${owner}/${project.path}.git`,
htmlUrl: project.web_url || `${this.baseUrl}/${owner}/${project.path}`,
isPrivate: project.visibility !== "public",
archived: !!project.archived,
created,
};
}
async getRepository(owner: string, name: string): Promise<PushTargetRepository | null> {
try {
const { data } = await targetFetch<GitLabProject>(
`${this.apiRoot}/projects/${this.projectPath(owner, name)}`,
{ headers: this.headers() }
);
// A deleted project on an instance with delayed deletion is renamed and
// kept for a while, and its old path redirects to it. That is not a
// repository we can push to; treat it as gone.
if (data.marked_for_deletion_at || data.marked_for_deletion_on) return null;
const requested = `${owner.replace(/^\/+|\/+$/g, "")}/${name}`.toLowerCase();
if (data.path_with_namespace && data.path_with_namespace.toLowerCase() !== requested) return null;
return this.toRepository(data, false);
} catch (error) {
if (isTargetNotFound(error)) return null;
throw error;
}
}
private ownsNamespace(owner: string): boolean {
return owner.trim().toLowerCase() === this.connection.username.trim().toLowerCase();
}
/** Find the group for a path, creating a top level group when it is missing. */
private async resolveGroupId(ownerPath: string): Promise<number> {
try {
const { data } = await targetFetch<GitLabNamespace>(
`${this.apiRoot}/namespaces/${encodeURIComponent(ownerPath)}`,
{ headers: this.headers() }
);
if (data?.id) return data.id;
} catch (error) {
if (!isTargetNotFound(error)) throw error;
}
if (ownerPath.includes("/")) {
throw new PushTargetError(
`GitLab group "${ownerPath}" was not found. Nested groups are not created automatically; create it first.`,
404
);
}
try {
const { data } = await targetFetch<GitLabNamespace>(`${this.apiRoot}/groups`, {
method: "POST",
headers: this.headers(),
body: { name: ownerPath, path: ownerPath, visibility: "private" },
});
if (!data?.id) throw new PushTargetError(`GitLab did not return the group it created for "${ownerPath}".`);
return data.id;
} catch (error) {
if (error instanceof PushTargetError) {
throw new PushTargetError(
`GitLab group "${ownerPath}" was not found and could not be created: ${error.message}`,
error.status,
error.url
);
}
throw error;
}
}
async ensureRepository(input: EnsureRepositoryInput): Promise<PushTargetRepository> {
const existing = await this.getRepository(input.owner, input.name);
if (existing) return existing;
const body: Record<string, unknown> = {
name: input.name,
path: input.name,
visibility: input.isPrivate ? "private" : "public",
description: (input.description || "").slice(0, MAX_DESCRIPTION) || undefined,
initialize_with_readme: false,
issues_access_level: "disabled",
wiki_access_level: "disabled",
};
if (!this.ownsNamespace(input.owner)) {
body.namespace_id = await this.resolveGroupId(input.owner);
}
try {
const { data } = await targetFetch<GitLabProject>(`${this.apiRoot}/projects`, {
method: "POST",
headers: this.headers(),
body,
});
return this.toRepository(data, true);
} catch (error) {
if (error instanceof PushTargetError && error.status === 400) {
// "has already been taken": lost a race with another run.
const again = await this.getRepository(input.owner, input.name);
if (again) return again;
}
throw error;
}
}
async setDefaultBranch(owner: string, name: string, branch: string): Promise<void> {
await targetFetch(`${this.apiRoot}/projects/${this.projectPath(owner, name)}`, {
method: "PUT",
headers: this.headers(),
body: { default_branch: branch },
});
}
async archiveRepository(owner: string, name: string): Promise<void> {
await targetFetch(`${this.apiRoot}/projects/${this.projectPath(owner, name)}/archive`, {
method: "POST",
headers: this.headers(),
});
}
async deleteRepository(owner: string, name: string): Promise<void> {
try {
await targetFetch(`${this.apiRoot}/projects/${this.projectPath(owner, name)}`, {
method: "DELETE",
headers: this.headers(),
});
} catch (error) {
if (isTargetNotFound(error)) return;
throw error;
}
}
}
+96
View File
@@ -0,0 +1,96 @@
/**
* fetch wrapper for the target adapters: JSON in, JSON out, PushTargetError
* on a non-2xx status. Built on the global fetch so tests can swap it.
*/
import { PushTargetError } from "./types";
export const DEFAULT_TARGET_TIMEOUT_MS = 30_000;
export interface TargetFetchInit {
method?: string;
headers?: Record<string, string>;
body?: unknown;
timeoutMs?: number;
}
export interface TargetFetchResult<T> {
data: T;
status: number;
}
export async function targetFetch<T>(
url: string,
init: TargetFetchInit = {}
): Promise<TargetFetchResult<T>> {
const { method = "GET", headers = {}, body, timeoutMs = DEFAULT_TARGET_TIMEOUT_MS } = init;
const response = await fetch(url, {
method,
headers: {
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
...headers,
},
body: body !== undefined ? JSON.stringify(body) : undefined,
signal: AbortSignal.timeout(timeoutMs),
});
if (!response.ok) {
let text = "";
try {
text = await response.text();
} catch {
// The status carries the meaning; the body is for the message only.
}
throw new PushTargetError(
`${method} ${redact(url)} failed with status ${response.status}${
summarize(text) ? `: ${summarize(text)}` : ""
}`,
response.status,
url
);
}
if (response.status === 204 || response.headers.get("content-length") === "0") {
return { data: undefined as T, status: response.status };
}
const text = await response.text();
if (!text) return { data: undefined as T, status: response.status };
return { data: JSON.parse(text) as T, status: response.status };
}
export function isTargetNotFound(error: unknown): boolean {
return error instanceof PushTargetError && error.status === 404;
}
function summarize(body: string): string {
if (!body) return "";
try {
const parsed = JSON.parse(body) as { message?: unknown; error?: unknown; errors?: unknown };
const message = parsed.message ?? parsed.error;
if (typeof message === "string") {
const errors = Array.isArray(parsed.errors)
? parsed.errors
.map((entry) =>
typeof entry === "string"
? entry
: entry && typeof entry === "object" && "message" in entry
? String((entry as { message: unknown }).message)
: ""
)
.filter(Boolean)
.join("; ")
: "";
return `${message}${errors ? ` (${errors})` : ""}`.slice(0, 300);
}
if (message && typeof message === "object") return JSON.stringify(message).slice(0, 300);
} catch {
// Not JSON.
}
return body.slice(0, 300);
}
/** Drop the query string from a URL before it lands in an error message. */
function redact(url: string): string {
return url.split("?")[0];
}
+19
View File
@@ -0,0 +1,19 @@
import type { PushDestinationKind } from "@/lib/destination-kinds";
import { GitHubPushTarget } from "./github";
import { GitLabPushTarget } from "./gitlab";
import type { PushTarget, PushTargetConnection } from "./types";
export * from "./types";
export { GitHubPushTarget, githubApiRootFor } from "./github";
export { GitLabPushTarget } from "./gitlab";
export { isTargetNotFound } from "./http";
export function createPushTarget(kind: PushDestinationKind, connection: PushTargetConnection): PushTarget {
switch (kind) {
case "gitlab":
return new GitLabPushTarget(connection);
case "github":
default:
return new GitHubPushTarget(connection);
}
}
+236
View File
@@ -0,0 +1,236 @@
/**
* The GitHub and GitLab target adapters against a mocked fetch.
*/
import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";
import { GitHubPushTarget, githubApiRootFor } from "./github";
import { GitLabPushTarget } from "./gitlab";
import { createPushTarget } from "./index";
import { PushTargetError } from "./types";
interface Call {
method: string;
url: string;
headers: Record<string, string>;
body: unknown;
}
let calls: Call[] = [];
let routes: Array<{ method: string; test: (url: string) => boolean; reply: (call: Call) => Response }> = [];
const originalFetch = globalThis.fetch;
function jsonResponse(data: unknown, status = 200): Response {
return new Response(JSON.stringify(data), { status, headers: { "Content-Type": "application/json" } });
}
function on(method: string, matcher: string | RegExp, reply: (call: Call) => Response) {
routes.push({
method,
test: (url) => (typeof matcher === "string" ? url === matcher : matcher.test(url)),
reply,
});
}
beforeEach(() => {
calls = [];
routes = [];
globalThis.fetch = mock(async (input: RequestInfo | URL, init?: RequestInit) => {
const url = typeof input === "string" ? input : input instanceof URL ? input.toString() : input.url;
const method = (init?.method ?? "GET").toUpperCase();
const headers = Object.fromEntries(new Headers(init?.headers as HeadersInit).entries());
const body = typeof init?.body === "string" ? JSON.parse(init.body) : undefined;
const call = { method, url, headers, body };
calls.push(call);
const route = routes.find((r) => r.method === method && r.test(url));
if (!route) return jsonResponse({ message: `no route for ${method} ${url}` }, 404);
return route.reply(call);
}) as unknown as typeof globalThis.fetch;
});
afterEach(() => {
globalThis.fetch = originalFetch;
});
describe("GitHub push target", () => {
const target = new GitHubPushTarget({ url: "https://github.com", username: "octocat", token: "gh-secret" });
test("uses api.github.com for github.com and /api/v3 for an enterprise host", () => {
expect(githubApiRootFor("https://github.com")).toBe("https://api.github.com");
expect(githubApiRootFor("https://ghe.example.com/")).toBe("https://ghe.example.com/api/v3");
});
test("testConnection returns the token's login and sends the bearer token", async () => {
on("GET", "https://api.github.com/user", () => jsonResponse({ login: "octocat" }));
const identity = await target.testConnection();
expect(identity.login).toBe("octocat");
expect(calls[0].headers.authorization).toBe("Bearer gh-secret");
});
test("ensureRepository returns an existing repository without creating one", async () => {
on("GET", "https://api.github.com/repos/octocat/tool", () =>
jsonResponse({ name: "tool", owner: { login: "octocat" }, private: true, archived: false, clone_url: "https://github.com/octocat/tool.git", html_url: "https://github.com/octocat/tool" })
);
const repo = await target.ensureRepository({ owner: "octocat", name: "tool", isPrivate: true });
expect(repo.created).toBe(false);
expect(repo.pushUrl).toBe("https://github.com/octocat/tool.git");
expect(calls.map((c) => c.method)).toEqual(["GET"]);
});
test("ensureRepository creates under the user when the owner is the token's account", async () => {
on("GET", "https://api.github.com/repos/Octocat/tool", () => jsonResponse({ message: "Not Found" }, 404));
on("POST", "https://api.github.com/user/repos", (call) =>
jsonResponse({ name: "tool", owner: { login: "octocat" }, private: call.body && (call.body as { private: boolean }).private, clone_url: "https://github.com/octocat/tool.git" })
);
const repo = await target.ensureRepository({ owner: "Octocat", name: "tool", isPrivate: true, description: "d" });
expect(repo.created).toBe(true);
expect(repo.isPrivate).toBe(true);
const create = calls.find((c) => c.method === "POST")!;
expect(create.body).toMatchObject({ name: "tool", private: true, has_issues: false, auto_init: false });
});
test("ensureRepository creates under the organization otherwise, and explains a missing one", async () => {
on("GET", "https://api.github.com/repos/acme/tool", () => jsonResponse({}, 404));
on("POST", "https://api.github.com/orgs/acme/repos", () =>
jsonResponse({ name: "tool", owner: { login: "acme" }, clone_url: "https://github.com/acme/tool.git" })
);
const repo = await target.ensureRepository({ owner: "acme", name: "tool", isPrivate: false });
expect(repo.owner).toBe("acme");
expect(repo.created).toBe(true);
calls = [];
routes = [];
on("GET", "https://api.github.com/repos/missing/tool", () => jsonResponse({}, 404));
on("POST", "https://api.github.com/orgs/missing/repos", () => jsonResponse({ message: "Not Found" }, 404));
await expect(target.ensureRepository({ owner: "missing", name: "tool", isPrivate: false })).rejects.toThrow(
/organization "missing" was not found/
);
});
test("a 422 on create re-reads the repository (lost race)", async () => {
let reads = 0;
on("GET", "https://api.github.com/repos/octocat/tool", () => {
reads += 1;
return reads === 1 ? jsonResponse({}, 404) : jsonResponse({ name: "tool", owner: { login: "octocat" }, clone_url: "u" });
});
on("POST", "https://api.github.com/user/repos", () => jsonResponse({ message: "name already exists" }, 422));
const repo = await target.ensureRepository({ owner: "octocat", name: "tool", isPrivate: false });
expect(repo.created).toBe(false);
});
test("archive patches archived, delete explains a missing scope and tolerates 404", async () => {
on("PATCH", "https://api.github.com/repos/octocat/tool", () => jsonResponse({ archived: true }));
await target.archiveRepository("octocat", "tool");
expect(calls[0].body).toEqual({ archived: true });
on("DELETE", "https://api.github.com/repos/octocat/tool", () => jsonResponse({ message: "Must have admin rights" }, 403));
await expect(target.deleteRepository("octocat", "tool")).rejects.toThrow(/delete_repo/);
routes = [];
on("DELETE", "https://api.github.com/repos/octocat/gone", () => jsonResponse({}, 404));
await expect(target.deleteRepository("octocat", "gone")).resolves.toBeUndefined();
});
test("push credentials use the account name and the token", () => {
expect(target.pushCredentials()).toEqual({ username: "octocat", token: "gh-secret" });
expect(new GitHubPushTarget({ url: "https://github.com", username: "", token: "t" }).pushCredentials().username).toBe(
"x-access-token"
);
});
});
describe("GitLab push target", () => {
const target = new GitLabPushTarget({ url: "https://gitlab.com", username: "jane", token: "gl-secret" });
test("testConnection sends the private token and reads the username", async () => {
on("GET", "https://gitlab.com/api/v4/user", () => jsonResponse({ username: "jane" }));
on("GET", "https://gitlab.com/api/v4/version", () => jsonResponse({ version: "17.4.0" }));
const identity = await target.testConnection();
expect(identity).toEqual({ login: "jane", label: "GitLab 17.4.0" });
expect(calls[0].headers["private-token"]).toBe("gl-secret");
});
test("ensureRepository looks projects up by encoded path and creates them in a group", async () => {
on("GET", "https://gitlab.com/api/v4/projects/acme%2Ftool", () => jsonResponse({ message: "404 Project Not Found" }, 404));
on("GET", "https://gitlab.com/api/v4/namespaces/acme", () => jsonResponse({ id: 42, full_path: "acme", kind: "group" }));
on("POST", "https://gitlab.com/api/v4/projects", (call) =>
jsonResponse({
id: 7,
name: "tool",
path: "tool",
namespace: { full_path: "acme" },
visibility: (call.body as { visibility: string }).visibility,
http_url_to_repo: "https://gitlab.com/acme/tool.git",
web_url: "https://gitlab.com/acme/tool",
})
);
const repo = await target.ensureRepository({ owner: "acme", name: "tool", isPrivate: true });
expect(repo.created).toBe(true);
expect(repo.isPrivate).toBe(true);
expect(repo.pushUrl).toBe("https://gitlab.com/acme/tool.git");
const create = calls.find((c) => c.method === "POST")!;
expect(create.body).toMatchObject({ name: "tool", path: "tool", namespace_id: 42, visibility: "private" });
});
test("a missing top level group is created; a missing nested group is not", async () => {
on("GET", "https://gitlab.com/api/v4/projects/newgroup%2Ftool", () => jsonResponse({}, 404));
on("GET", "https://gitlab.com/api/v4/namespaces/newgroup", () => jsonResponse({}, 404));
on("POST", "https://gitlab.com/api/v4/groups", () => jsonResponse({ id: 99, full_path: "newgroup" }));
on("POST", "https://gitlab.com/api/v4/projects", () =>
jsonResponse({ id: 8, name: "tool", path: "tool", namespace: { full_path: "newgroup" }, http_url_to_repo: "u", web_url: "w" })
);
const repo = await target.ensureRepository({ owner: "newgroup", name: "tool", isPrivate: false });
expect(repo.created).toBe(true);
expect(calls.find((c) => c.url.endsWith("/groups"))!.body).toMatchObject({ path: "newgroup", visibility: "private" });
routes = [];
on("GET", "https://gitlab.com/api/v4/projects/parent%2Fchild%2Ftool", () => jsonResponse({}, 404));
on("GET", "https://gitlab.com/api/v4/namespaces/parent%2Fchild", () => jsonResponse({}, 404));
await expect(target.ensureRepository({ owner: "parent/child", name: "tool", isPrivate: false })).rejects.toThrow(
/Nested groups are not created automatically/
);
});
test("projects under the token's own user need no namespace lookup", async () => {
on("GET", "https://gitlab.com/api/v4/projects/jane%2Ftool", () => jsonResponse({}, 404));
on("POST", "https://gitlab.com/api/v4/projects", () =>
jsonResponse({ id: 9, name: "tool", path: "tool", namespace: { full_path: "jane" }, http_url_to_repo: "u", web_url: "w" })
);
await target.ensureRepository({ owner: "jane", name: "tool", isPrivate: false });
expect(calls.some((c) => c.url.includes("/namespaces/"))).toBe(false);
expect((calls.find((c) => c.method === "POST")!.body as { namespace_id?: number }).namespace_id).toBeUndefined();
});
test("archive posts to /archive and delete tolerates a missing project", async () => {
on("POST", "https://gitlab.com/api/v4/projects/acme%2Ftool/archive", () => jsonResponse({ archived: true }));
await target.archiveRepository("acme", "tool");
on("DELETE", "https://gitlab.com/api/v4/projects/acme%2Fgone", () => jsonResponse({}, 404));
await expect(target.deleteRepository("acme", "gone")).resolves.toBeUndefined();
on("DELETE", "https://gitlab.com/api/v4/projects/acme%2Flocked", () => jsonResponse({ message: "403 Forbidden" }, 403));
await expect(target.deleteRepository("acme", "locked")).rejects.toBeInstanceOf(PushTargetError);
});
test("push credentials use the oauth2 user", () => {
expect(target.pushCredentials()).toEqual({ username: "oauth2", token: "gl-secret" });
});
test("a project waiting in delayed deletion, or reached through a renamed path, counts as missing", async () => {
on("GET", "https://gitlab.com/api/v4/projects/acme%2Fold", () =>
jsonResponse({ id: 1, name: "old", path: "old", path_with_namespace: "acme/old", marked_for_deletion_at: "2026-09-02" })
);
expect(await target.getRepository("acme", "old")).toBeNull();
on("GET", "https://gitlab.com/api/v4/projects/acme%2Fmoved", () =>
jsonResponse({ id: 2, name: "moved", path: "moved-deletion_scheduled-1", path_with_namespace: "acme/moved-deletion_scheduled-1" })
);
expect(await target.getRepository("acme", "moved")).toBeNull();
on("GET", "https://gitlab.com/api/v4/projects/acme%2Flive", () =>
jsonResponse({ id: 3, name: "live", path: "live", path_with_namespace: "acme/live" })
);
expect((await target.getRepository("acme", "live"))?.name).toBe("live");
});
});
describe("createPushTarget", () => {
test("builds the adapter for each kind", () => {
expect(createPushTarget("github", { url: "https://github.com", username: "u", token: "t" }).kind).toBe("github");
expect(createPushTarget("gitlab", { url: "https://gitlab.com", username: "u", token: "t" }).kind).toBe("gitlab");
});
});
+71
View File
@@ -0,0 +1,71 @@
/**
* A push target is the host that receives `git push --mirror`.
*
* The engine only needs a handful of things from it: make sure the
* repository exists, tell it where to push and with which credentials, and
* archive or delete when the cleanup service asks. Everything else about
* the host (issues, releases, wiki) is out of the engine's scope.
*/
import type { GitCredentials } from "../git";
export interface PushTargetRepository {
owner: string;
name: string;
/** http(s) URL git pushes to. Credentials are supplied separately. */
pushUrl: string;
htmlUrl: string;
isPrivate: boolean;
archived: boolean;
/** True when this call created the repository. */
created: boolean;
}
export interface EnsureRepositoryInput {
owner: string;
name: string;
isPrivate: boolean;
description?: string | null;
}
export interface PushTargetIdentity {
/** The login the token belongs to. */
login: string;
/** Human readable, e.g. "GitHub Enterprise 3.12" or "GitLab 17.4". */
label?: string;
}
export class PushTargetError extends Error {
constructor(
message: string,
public readonly status?: number,
public readonly url?: string
) {
super(message);
this.name = "PushTargetError";
}
}
export interface PushTarget {
readonly kind: "github" | "gitlab" | "local";
/** Base URL of the host, e.g. https://github.com. */
readonly baseUrl: string;
/** Credentials for the push. Null for targets that need none (local files). */
pushCredentials(): GitCredentials | null;
/** Check the token: returns who it belongs to or throws. */
testConnection(): Promise<PushTargetIdentity>;
getRepository(owner: string, name: string): Promise<PushTargetRepository | null>;
/** Create the repository when missing. Never touches an existing one. */
ensureRepository(input: EnsureRepositoryInput): Promise<PushTargetRepository>;
archiveRepository(owner: string, name: string): Promise<void>;
deleteRepository(owner: string, name: string): Promise<void>;
/** Best effort: make the target open on the same branch as the source. */
setDefaultBranch?(owner: string, name: string, branch: string): Promise<void>;
}
/** The connection details a target is built from. */
export interface PushTargetConnection {
url: string;
/** The account the token belongs to. */
username: string;
token: string;
}
+3 -3
View File
@@ -7,7 +7,7 @@ import { findInterruptedJobs, resumeInterruptedJob } from './helpers';
import { resetStuckMirrorStatuses } from './stuck-status-recovery';
import { db, repositories, organizations, mirrorJobs, configs } from './db';
import { eq, and, lt, inArray, sql } from 'drizzle-orm';
import { mirrorGithubRepoToGitea, syncGiteaRepo } from './gitea';
import { mirrorRepositoryToDestination, syncRepositoryOnDestination } from './mirror-dispatch';
import { createGitHubClient } from './github';
import type { Octokit } from '@octokit/rest';
import { resolveSourceProviderKind } from './source-providers';
@@ -303,7 +303,7 @@ async function recoverMirrorJob(job: any, remainingItemIds: string[]) {
mirroredLocation: repo.mirroredLocation || "",
};
await mirrorGithubRepoToGitea({
await mirrorRepositoryToDestination({
octokit,
repository: repoData,
config,
@@ -411,7 +411,7 @@ async function recoverSyncJob(job: any, remainingItemIds: string[]) {
};
// Sync the repository
await syncGiteaRepo({
await syncRepositoryOnDestination({
config,
repository: repoData,
});
+11
View File
@@ -2,6 +2,7 @@ import { v4 as uuidv4 } from 'uuid';
import type { GitRepo } from '@/types/Repository';
import { repositories } from '@/lib/db/schema';
import { getRepositorySource } from '@/lib/source-providers/kinds';
import type { RepositoryDestination } from '@/lib/destination-kinds';
export type RepoInsert = typeof repositories.$inferInsert;
@@ -80,3 +81,13 @@ export function calcBatchSizeForInsert(columnCount: number, maxParams = 999): nu
const effectiveMax = Math.max(1, maxParams - safety);
return Math.max(1, Math.floor(effectiveMax / columnCount));
}
/**
* The destination columns for a new repository row: where its mirror will
* live, taken from the destination that is configured now.
*/
export function repositoryDestinationColumns(
destination: RepositoryDestination
): Pick<RepoInsert, 'destinationProvider' | 'destinationUrl'> {
return { destinationProvider: destination.provider, destinationUrl: destination.url };
}
+17 -2
View File
@@ -13,6 +13,7 @@ import {
isRepositoryFromConfiguredSource,
} from '@/lib/source-providers';
import { createGiteaClient, deleteGiteaRepo, archiveGiteaRepo, getGiteaRepoOwnerAsync, checkRepoLocation } from '@/lib/gitea';
import { usesPushEngine } from '@/lib/destination-connection';
import { getDecryptedGiteaToken } from '@/lib/utils/config-encryption';
import { publishEvent } from '@/lib/events';
import { isMirrorableGitHubRepo } from '@/lib/repo-eligibility';
@@ -370,7 +371,11 @@ async function handleOrphanedRepository(
updatedAt: new Date(),
};
if (plan.gitea === 'archive') {
if (plan.gitea === 'archive' && usesPushEngine(config)) {
console.log(`[Repository Cleanup] Archiving orphaned repository ${repoFullName} on the push target`);
const { archiveOnPushTarget } = await import('@/lib/push-engine/cleanup');
await archiveOnPushTarget({ config, repository: repo });
} else if (plan.gitea === 'archive') {
console.log(`[Repository Cleanup] Archiving orphaned repository ${repoFullName} in Gitea`);
const { giteaClient, giteaRepoName, giteaOwner: expectedOwner } = await resolveGiteaTarget(config, repo);
let giteaOwner = expectedOwner;
@@ -429,7 +434,11 @@ async function handleOrphanedRepository(
},
});
} else if (action === 'delete') {
if (plan.gitea === 'delete') {
if (plan.gitea === 'delete' && usesPushEngine(config)) {
console.log(`[Repository Cleanup] Deleting orphaned repository ${repoFullName} from the push target`);
const { deleteOnPushTarget } = await import('@/lib/push-engine/cleanup');
await deleteOnPushTarget({ config, repository: repo });
} else if (plan.gitea === 'delete') {
console.log(`[Repository Cleanup] Deleting orphaned repository ${repoFullName} from Gitea`);
const { giteaClient, giteaOwner, giteaRepoName } = await resolveGiteaTarget(config, repo);
await deleteGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
@@ -440,6 +449,12 @@ async function handleOrphanedRepository(
// Delete from database
await db.delete(repositories).where(eq(repositories.id, repo.id));
if (usesPushEngine(config)) {
// The bare clone has no row to belong to any more.
const { removeCloneForRepository } = await import('@/lib/push-engine/cleanup');
await removeCloneForRepository({ repository: repo });
}
// Create event
await publishEvent({
userId: config.userId,
+7 -6
View File
@@ -6,7 +6,8 @@
import { db, configs, repositories } from '@/lib/db';
import { eq, and, or } from 'drizzle-orm';
import { syncGiteaRepo, mirrorGithubRepoToGitea } from '@/lib/gitea';
import { mirrorRepositoryToDestination, syncRepositoryOnDestination } from '@/lib/mirror-dispatch';
import { usesPushEngine } from '@/lib/destination-connection';
import { getDecryptedGitHubToken } from '@/lib/utils/config-encryption';
import { formatDuration } from '@/lib/utils/duration-parser';
import type { Repository } from '@/lib/db/schema';
@@ -277,7 +278,7 @@ async function runScheduledSync(config: any): Promise<void> {
// re-create loop spawns suffixed duplicates (#315). The create
// path also reuses now, but routing to sync here avoids a
// wasted migrate attempt and keeps recovery cheap.
if (repo.status === 'failed' && repository.mirroredLocation) {
if (repo.status === 'failed' && repository.mirroredLocation && !usesPushEngine(config)) {
const { findExistingMirror } = await import('@/lib/utils/mirror-source-match');
const existing = await findExistingMirror({
repository,
@@ -286,13 +287,13 @@ async function runScheduledSync(config: any): Promise<void> {
candidateName: repository.name,
});
if (existing) {
await syncGiteaRepo({ config, repository });
await syncRepositoryOnDestination({ config, repository });
console.log(`[Scheduler] Re-synced failed repository with live mirror: ${repo.fullName}`);
return;
}
}
await mirrorGithubRepoToGitea({ octokit, repository, config });
await mirrorRepositoryToDestination({ octokit, repository, config });
console.log(`[Scheduler] Auto-mirrored repository: ${repo.fullName}`);
} catch (error) {
console.error(`[Scheduler] Failed to auto-mirror repository ${repo.fullName}:`, error);
@@ -426,7 +427,7 @@ async function syncSingleRepository(config: any, repo: any): Promise<void> {
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
await syncGiteaRepo({ config, repository });
await syncRepositoryOnDestination({ config, repository });
console.log(`[Scheduler] Successfully synced repository ${repo.fullName}`);
} catch (error) {
console.error(`[Scheduler] Failed to sync repository ${repo.fullName}:`, error);
@@ -682,7 +683,7 @@ async function performInitialAutoStart(): Promise<void> {
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
await mirrorGithubRepoToGitea({
await mirrorRepositoryToDestination({
octokit,
repository,
config
+45
View File
@@ -2,6 +2,7 @@ import type { Config } from "@/types/config";
import type { MirrorOverrides, Repository } from "@/lib/db/schema";
import { mirrorOverridesSchema } from "@/lib/db/schema";
import { normalizeSourceProviderKind } from "@/lib/source-providers/kinds";
import { isPushDestinationKind, normalizeDestinationProviderKind } from "@/lib/destination-kinds";
/**
* The mirror option flags that can be overridden per organization and per
@@ -139,6 +140,29 @@ export const GITHUB_ONLY_METADATA_KEYS = [
"mirrorMilestones",
] as const satisfies readonly MirrorOverrideKey[];
/**
* Flags that cannot take effect on a push target.
*
* GitHub and GitLab destinations receive `git push` of branches and tags
* and nothing else: no wiki clone, no LFS transfer, no issues, releases,
* labels or milestones. The resolver forces all of these off for them.
*/
export const PUSH_TARGET_CLAMPED_KEYS = [
"lfs",
"wiki",
"mirrorReleases",
"mirrorMetadata",
"mirrorIssues",
"mirrorPullRequests",
"mirrorLabels",
"mirrorMilestones",
] as const satisfies readonly MirrorOverrideKey[];
/** True when a config's destination is served by the push engine. */
export function isPushDestinationConfig(config: { giteaConfig?: { provider?: unknown } | null } | null | undefined): boolean {
return isPushDestinationKind(normalizeDestinationProviderKind(config?.giteaConfig?.provider));
}
/**
* Flags surfaced in the override UI.
*
@@ -227,6 +251,7 @@ export const MIRROR_GATING_REASONS = {
"Starred repos mirror code only (Advanced Options > starred code only)",
labelsFollowIssues: "Issues mirroring already syncs labels",
releasesOff: "Releases are not being mirrored",
pushTarget: "Not available for GitHub and GitLab destinations: only branches and tags are pushed",
} as const;
/** key -> reason it cannot take effect. Absent key means editable. */
@@ -258,14 +283,27 @@ export function getMirrorOverrideGating({
isStarred,
starredCodeOnly,
effective,
destinationProvider,
}: {
targetKind: "repository" | "organization";
isStarred?: boolean;
starredCodeOnly?: boolean;
effective: InheritedMirrorOptions;
/** The configured destination kind; push targets disable every toggle. */
destinationProvider?: unknown;
}): MirrorOverrideGating {
const gating: MirrorOverrideGating = {};
// A push target gets branches and tags only, whatever any tier asks for.
if (isPushDestinationKind(normalizeDestinationProviderKind(destinationProvider))) {
for (const key of PUSH_TARGET_CLAMPED_KEYS) {
gating[key] = MIRROR_GATING_REASONS.pushTarget;
}
gating.releaseLimit = MIRROR_GATING_REASONS.pushTarget;
gating.releaseAssetLimit = MIRROR_GATING_REASONS.pushTarget;
return gating;
}
// Organizations are never starred, so the clamp cannot apply to them.
const starredClampApplies =
targetKind === "repository" && !!isStarred && !!starredCodeOnly;
@@ -477,6 +515,13 @@ export function resolveMirrorOptions({
}
}
// A push target receives branches and tags only.
if (isPushDestinationConfig(config)) {
for (const key of PUSH_TARGET_CLAMPED_KEYS) {
resolved[key] = false;
}
}
return resolved;
}
+12
View File
@@ -6,6 +6,7 @@ import type { Config } from "@/types/config";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
import { reconcileDestination } from "@/lib/destination-reconcile-service";
import { usesPushEngine } from "@/lib/destination-connection";
const bodySchema = z.object({
dryRun: z.boolean().optional().default(true),
@@ -70,6 +71,17 @@ export const POST: APIRoute = async ({ request, locals }) => {
});
}
if (usesPushEngine(config)) {
return jsonResponse({
data: {
success: false,
error:
"Reconcile compares a Gitea or Forgejo destination with the database. GitHub and GitLab destinations are pushed by this app and have nothing to reconcile.",
},
status: 400,
});
}
const result = await reconcileDestination(config as unknown as Config, parsed.data);
return jsonResponse({
+22
View File
@@ -6,6 +6,12 @@ import {
normalizeSourceProviderKind,
} from "@/lib/source-providers/kinds";
import { evaluateConfigChange, loadConfigLocks } from "@/lib/config-locks";
import {
DESTINATION_PROVIDER_DEFAULT_URLS,
isDestinationProviderKind,
isPushDestinationKind,
normalizeDestinationProviderKind,
} from "@/lib/destination-kinds";
import { db, configs, users } from "@/lib/db";
import { v4 as uuidv4 } from "uuid";
import { eq, sql } from "drizzle-orm";
@@ -92,6 +98,22 @@ export const POST: APIRoute = async ({ request, locals }) => {
}
}
// Validate the destination provider. GitHub and GitLab targets default to
// their public instance when no URL is given.
if (giteaConfig.provider !== undefined && !isDestinationProviderKind(giteaConfig.provider)) {
return new Response(
JSON.stringify({ success: false, message: "Unknown destination provider." }),
{ status: 400, headers: { "Content-Type": "application/json" } }
);
}
const destinationProvider = normalizeDestinationProviderKind(giteaConfig.provider);
if (
isPushDestinationKind(destinationProvider) &&
!(typeof giteaConfig.url === "string" && giteaConfig.url.trim())
) {
giteaConfig.url = DESTINATION_PROVIDER_DEFAULT_URLS[destinationProvider];
}
// Validate the source provider and, for GitLab and Gitea sources, the instance URL
if (githubConfig.provider !== undefined && !isSourceProviderKind(githubConfig.provider)) {
return new Response(
+74 -69
View File
@@ -1,6 +1,14 @@
import type { APIRoute } from 'astro';
import { httpGet, HttpError } from '@/lib/http-client';
import { createSecureErrorResponse } from '@/lib/utils';
import {
DESTINATION_PROVIDER_LABELS,
isPushDestinationKind,
normalizeDestinationBaseUrl,
normalizeDestinationProviderKind,
type PushDestinationKind,
} from '@/lib/destination-kinds';
import { createPushTarget, PushTargetError } from '@/lib/push-engine/targets';
// Forgejo reports `15.0.0+gitea-1.22.0`; pure Gitea reports just `1.22.0`.
// Forgejo < 15.0.0 has a known bug where pull-mirror credentials sent via
@@ -21,24 +29,70 @@ function parseServerInfo(versionString: string) {
return { type: 'gitea' as const, version: versionString, raw: versionString, hasMirrorCredBug: false };
}
function json(data: unknown, status: number): Response {
return new Response(JSON.stringify(data), {
status,
headers: { 'Content-Type': 'application/json' },
});
}
/**
* GitHub and GitLab destinations are push targets: the check is whether the
* token identifies a user there, and that it belongs to the account named
* in the form when one is given.
*/
async function testPushTarget(
kind: PushDestinationKind,
{ url, token, username }: { url?: string; token?: string; username?: string }
): Promise<Response> {
const label = DESTINATION_PROVIDER_LABELS[kind];
if (!token) {
return json({ success: false, message: `${label} token is required` }, 400);
}
const baseUrl = normalizeDestinationBaseUrl(url, kind);
try {
const target = createPushTarget(kind, { url: baseUrl, username: username || '', token });
const identity = await target.testConnection();
if (username && identity.login.toLowerCase() !== String(username).toLowerCase()) {
return json({ success: false, message: `Token belongs to ${identity.login}, not ${username}` }, 400);
}
const version = identity.label ?? label;
return json(
{
success: true,
message: `Successfully connected to ${version} as ${identity.login}`,
user: { login: identity.login },
serverInfo: { type: kind, version, raw: version, hasMirrorCredBug: false },
},
200
);
} catch (error) {
console.error(`${label} connection test failed:`, error);
if (error instanceof PushTargetError && error.status === 401) {
return json({ success: false, message: `Invalid ${label} token` }, 401);
}
if (error instanceof PushTargetError && error.status === 404) {
return json({ success: false, message: `${label} API endpoint not found. Please check the URL.` }, 404);
}
return createSecureErrorResponse(error, `${label} connection test`, 500);
}
}
export const POST: APIRoute = async ({ request }) => {
try {
const body = await request.json();
const { url, token, username } = body;
const provider = normalizeDestinationProviderKind(body.provider);
if (isPushDestinationKind(provider)) {
return testPushTarget(provider, { url, token, username });
}
if (!url || !token) {
return new Response(
JSON.stringify({
success: false,
message: 'Gitea URL and token are required',
}),
{
status: 400,
headers: {
'Content-Type': 'application/json',
},
}
);
return json({ success: false, message: 'Gitea URL and token are required' }, 400);
}
// Normalize the URL (remove trailing slash if present)
@@ -54,18 +108,7 @@ export const POST: APIRoute = async ({ request }) => {
// Verify that the authenticated user matches the provided username (if provided)
if (username && data.login !== username) {
return new Response(
JSON.stringify({
success: false,
message: `Token belongs to ${data.login}, not ${username}`,
}),
{
status: 400,
headers: {
'Content-Type': 'application/json',
},
}
);
return json({ success: false, message: `Token belongs to ${data.login}, not ${username}` }, 400);
}
let serverInfo: ReturnType<typeof parseServerInfo> | undefined;
@@ -80,8 +123,8 @@ export const POST: APIRoute = async ({ request }) => {
// Version probe is best-effort; older or non-standard servers may not expose it.
}
return new Response(
JSON.stringify({
return json(
{
success: true,
message: `Successfully connected to Gitea as ${data.login}`,
user: {
@@ -90,13 +133,8 @@ export const POST: APIRoute = async ({ request }) => {
avatar_url: data.avatar_url,
},
serverInfo,
}),
{
status: 200,
headers: {
'Content-Type': 'application/json',
},
}
},
200
);
} catch (error) {
console.error('Gitea connection test failed:', error);
@@ -104,45 +142,12 @@ export const POST: APIRoute = async ({ request }) => {
// Handle specific error types
if (error instanceof HttpError) {
if (error.status === 401) {
return new Response(
JSON.stringify({
success: false,
message: 'Invalid Gitea token',
}),
{
status: 401,
headers: {
'Content-Type': 'application/json',
},
}
);
return json({ success: false, message: 'Invalid Gitea token' }, 401);
} else if (error.status === 404) {
return new Response(
JSON.stringify({
success: false,
message: 'Gitea API endpoint not found. Please check the URL.',
}),
{
status: 404,
headers: {
'Content-Type': 'application/json',
},
}
);
return json({ success: false, message: 'Gitea API endpoint not found. Please check the URL.' }, 404);
} else if (error.status === 0) {
// Network error
return new Response(
JSON.stringify({
success: false,
message: 'Could not connect to Gitea server. Please check the URL.',
}),
{
status: 500,
headers: {
'Content-Type': 'application/json',
},
}
);
return json({ success: false, message: 'Could not connect to Gitea server. Please check the URL.' }, 500);
}
}
+10 -19
View File
@@ -5,11 +5,8 @@ import type { MirrorRepoRequest, MirrorRepoResponse } from "@/types/mirror";
import { db, configs, repositories } from "@/lib/db";
import { and, eq, inArray, sql } from "drizzle-orm";
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
import {
mirrorGithubRepoToGitea,
mirrorGitHubOrgRepoToGiteaOrg,
getGiteaRepoOwnerAsync,
} from "@/lib/gitea";
import { getGiteaRepoOwnerAsync } from "@/lib/gitea";
import { mirrorRepositoryToDestination } from "@/lib/mirror-dispatch";
import { createGitHubClient } from "@/lib/github";
import { getDecryptedGitHubToken } from "@/lib/utils/config-encryption";
import { processWithResilience } from "@/lib/utils/concurrency";
@@ -135,20 +132,14 @@ export const POST: APIRoute = async ({ request, locals }) => {
owner !== config.giteaConfig?.defaultOwner || // Different owner means org
mirrorStrategy === "single-org"; // Single-org strategy always uses org
if (shouldUseOrgMirror) {
await mirrorGitHubOrgRepoToGiteaOrg({
config,
octokit,
orgName: owner,
repository: repoData,
});
} else {
await mirrorGithubRepoToGitea({
octokit,
repository: repoData,
config,
});
}
// Gitea and Forgejo receive a pull mirror; GitHub and GitLab are
// pushed by the engine. The dispatcher picks the path.
await mirrorRepositoryToDestination({
config,
octokit,
repository: repoData,
orgName: shouldUseOrgMirror ? owner : undefined,
});
return repo;
},
+20 -26
View File
@@ -3,11 +3,8 @@ import { resolveSourceProviderKind } from "@/lib/source-providers";
import { db, configs, repositories } from "@/lib/db";
import { and, eq, inArray, sql } from "drizzle-orm";
import { getGiteaRepoOwnerAsync, isRepoPresentInGitea } from "@/lib/gitea";
import {
mirrorGithubRepoToGitea,
mirrorGitHubOrgRepoToGiteaOrg,
syncGiteaRepo,
} from "@/lib/gitea";
import { mirrorRepositoryToDestination, syncRepositoryOnDestination } from "@/lib/mirror-dispatch";
import { usesPushEngine } from "@/lib/destination-connection";
import { createGitHubClient } from "@/lib/github";
import { repoStatusEnum, repositoryVisibilityEnum } from "@/types/Repository";
import type { RetryRepoRequest, RetryRepoResponse } from "@/types/retry";
@@ -129,6 +126,14 @@ export const POST: APIRoute = async ({ request, locals }) => {
status: "imported",
});
if (usesPushEngine(config)) {
// A push target: the engine clones or fetches, then pushes, and
// creates the target repository itself when it is missing.
await mirrorRepositoryToDestination({ config, octokit, repository: repoData });
console.log(`Pushed repo: ${repo.name}`);
return repo;
}
// Determine if the repository exists in Gitea (with organization overrides)
let owner = await getGiteaRepoOwnerAsync({
config,
@@ -143,7 +148,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
if (present) {
// If the repository exists, sync it
await syncGiteaRepo({ config, repository: repoData });
await syncRepositoryOnDestination({ config, repository: repoData });
console.log(`Synced existing repo: ${repo.name}`);
} else {
// If the repository doesn't exist, mirror it
@@ -166,26 +171,15 @@ export const POST: APIRoute = async ({ request, locals }) => {
owner !== config.giteaConfig?.defaultOwner || // Different owner means org
mirrorStrategy === "single-org"; // Single-org strategy always uses org
if (shouldUseOrgMirror) {
await mirrorGitHubOrgRepoToGiteaOrg({
config,
octokit,
orgName: owner,
repository: {
...repoData,
status: repoStatusEnum.parse("imported"),
},
});
} else {
await mirrorGithubRepoToGitea({
config,
octokit,
repository: {
...repoData,
status: repoStatusEnum.parse("imported"),
},
});
}
await mirrorRepositoryToDestination({
config,
octokit,
repository: {
...repoData,
status: repoStatusEnum.parse("imported"),
},
orgName: shouldUseOrgMirror ? owner : undefined,
});
}
return repo;
+5 -3
View File
@@ -2,7 +2,9 @@ import type { APIRoute } from "astro";
import { db, configs, repositories } from "@/lib/db";
import { and, eq, or, sql } from "drizzle-orm";
import { repoStatusEnum, repositoryVisibilityEnum } from "@/types/Repository";
import { isRepoPresentInGitea, syncGiteaRepo } from "@/lib/gitea";
import { isRepoPresentInGitea } from "@/lib/gitea";
import { syncRepositoryOnDestination } from "@/lib/mirror-dispatch";
import { usesPushEngine } from "@/lib/destination-connection";
import type {
ScheduleSyncRepoRequest,
ScheduleSyncRepoResponse,
@@ -99,7 +101,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
for (const repo of repos) {
try {
// Only check Gitea presence if the repo failed previously
if (repo.status === "failed") {
if (repo.status === "failed" && !usesPushEngine(config)) {
const isPresent = await isRepoPresentInGitea({
config,
owner: repo.owner,
@@ -111,7 +113,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
}
}
await syncGiteaRepo({
await syncRepositoryOnDestination({
config,
repository: {
...repo,
+2 -2
View File
@@ -3,7 +3,7 @@ import type { MirrorRepoRequest } from "@/types/mirror";
import { db, configs, repositories } from "@/lib/db";
import { and, eq, inArray, sql } from "drizzle-orm";
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
import { syncGiteaRepo } from "@/lib/gitea";
import { syncRepositoryOnDestination } from "@/lib/mirror-dispatch";
import type { SyncRepoResponse } from "@/types/sync";
import { processWithResilience } from "@/lib/utils/concurrency";
import { createSecureErrorResponse } from "@/lib/utils";
@@ -99,7 +99,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
console.log(`Starting sync for repository: ${repo.name}`);
// Sync the repository
await syncGiteaRepo({
await syncRepositoryOnDestination({
config,
repository: repoData,
});
+22 -1
View File
@@ -3,6 +3,7 @@ import { db, repositories, mirrorJobs } from "@/lib/db";
import { eq, and, inArray } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
import { isPushDestinationKind } from "@/lib/destination-kinds";
export const DELETE: APIRoute = async (context) => {
try {
@@ -22,7 +23,14 @@ export const DELETE: APIRoute = async (context) => {
// Verify all repos belong to this user before deleting
const owned = await db
.select({ id: repositories.id })
.select({
id: repositories.id,
owner: repositories.owner,
name: repositories.name,
sourceProvider: repositories.sourceProvider,
sourceUrl: repositories.sourceUrl,
destinationProvider: repositories.destinationProvider,
})
.from(repositories)
.where(and(inArray(repositories.id, ids), eq(repositories.userId, userId)));
@@ -39,6 +47,19 @@ export const DELETE: APIRoute = async (context) => {
await tx.delete(repositories).where(and(inArray(repositories.id, ownedIds), eq(repositories.userId, userId)));
});
// Rows pushed by the engine leave a bare clone behind; drop it with the row.
const pushed = owned.filter((row) => isPushDestinationKind(row.destinationProvider));
if (pushed.length > 0) {
const { removeCloneForRepository } = await import("@/lib/push-engine/cleanup");
await Promise.all(
pushed.map((row) =>
removeCloneForRepository({ repository: row }).catch((error) =>
console.warn(`[Repositories] Could not remove the clone for ${row.owner}/${row.name}:`, error)
)
)
);
}
return new Response(
JSON.stringify({ success: true, deleted: ownedIds.length }),
{ status: 200, headers: { "Content-Type": "application/json" } }
+5 -2
View File
@@ -7,8 +7,11 @@ export type BackupStrategy = "disabled" | "always" | "on-force-push" | "block-on
export type ScheduleMode = "interval" | "clock";
/** Which host repositories are pulled from. "gitea" also covers Forgejo. */
export type SourceProvider = "github" | "gitlab" | "gitea";
/** Which host repositories are mirrored into. Same API, different label. */
export type DestinationProvider = "gitea" | "forgejo";
/**
* Which host repositories are mirrored into. Gitea and Forgejo are pull
* mirrors; GitHub and GitLab are push targets served by the push engine.
*/
export type DestinationProvider = "gitea" | "forgejo" | "github" | "gitlab";
export interface GiteaConfig {
/** Defaults to "gitea" when absent. */