mirror of
https://github.com/RayLabsHQ/gitea-mirror.git
synced 2026-09-28 03:21:53 +02:00
ARUNAVO RAYandGitHub
0f700c7197
* feat: per-repository and per-organization mirror option overrides Closes the gap reported in #361: a repository whose LFS fetch fails could not be mirrored at all, because LFS was a single global switch. Gitea runs the LFS fetch inside its own migration, so a failure aborts the whole migration with nothing to salvage. The only fix available to us is to stop asking for LFS on that repository. Mirror options now resolve across three tiers, per flag, most specific first: repository override, then organization override, then the global config. NULL at a tier means inherit, so overriding LFS on one repo leaves its other options alone. Adds resolveMirrorOptions(), which replaces 24 scattered reads of config.giteaConfig.* across three near-identical blocks: the two mirror paths in gitea.ts and the sync path in gitea-enhanced.ts. The third was easy to miss and mattered: without it, overrides would have been honored on first mirror and silently ignored on every scheduled sync afterwards. The starredCodeOnly clamp is folded into the resolver and deliberately outranks explicit overrides, preserving the existing behavior that starred repos mirror code only. Editing is on the objects themselves, via the existing three-dot menus on the Repositories and Organizations pages, not in Configuration, which keeps holding the global defaults. A repositories filter and a row badge make it possible to find which repos deviate. Malformed override JSON degrades to "inherit" rather than throwing, so a bad value can never break a mirror run. * fix(ui): disable mirror toggles that cannot take effect, and explain why A toggle the runtime will ignore should not look editable. Three cases were doing exactly that, so they now share one mechanism: getMirrorOverrideGating() returns a reason string per flag, and the dialog disables the control and prints that reason underneath. Starred clamp. When a repo is starred and starredCodeOnly is set, the resolver forces every metadata flag off regardless of the override. The dialog previously let the user set them anyway. The clamped set now lives in STARRED_CLAMPED_KEYS, shared by the resolver and the gating helper so the flags the UI disables cannot drift from the ones the runtime clamps. LFS is deliberately excluded from that set, since turning LFS off per repository is the point of #361, and a test pins that. Labels. shouldMirrorLabels is `mirrorLabels && !mirrorIssues` in both mirror paths, so labels cannot take effect while issues are mirrored. That gate reacts live to the in-progress edit rather than only the saved state. Inherit hint. For a repository the hint now reflects global -> org instead of global alone, via a new name-keyed GET /api/organizations/mirror-overrides that reuses loadOrganizationMirrorOverrides. Personal repos skip the fetch, a failed fetch degrades to the global values rather than blocking the dialog, and the hint is suppressed while in flight so it is never briefly wrong. Widens the UI to mirrorLabels and mirrorMilestones. mirrorMetadata stays out: no mirror path reads it, so a per-object override would resolve correctly and then do nothing. See the report for that finding. useGiteaConfig additionally returns advancedOptions so callers can read starredCodeOnly without a second request. * fix(ui): read inherited mirror flags from mirrorOptions, not giteaConfig /api/config does not return the mirror flags on giteaConfig. On the way out, mapDbToUiConfig reshapes them into a separate mirrorOptions object using different names (mirrorLFS, not lfs) and nesting the metadata flags under metadataComponents with short names (issues, not mirrorIssues). The dialog was written against the DB shape, so every lookup returned undefined. Coerced to false, that is indistinguishable from a real "off", which produced two symptoms: every inherit hint read "currently off" regardless of the actual global config, and the labels gate never fired because the effective issues value it keys on came from the same dead source. Adds mirrorOptionsToFlags() as the single conversion point, matching the derivation in mapUiToDbConfig exactly, including that mirrorMetadata is a master switch over the metadata components while lfs and mirrorReleases sit outside it. Both dialogs now go through it, and useGiteaConfig returns mirrorOptions alongside advancedOptions. Server-side mirroring was never affected: the resolver reads config straight from the DB, where the flags really do live on giteaConfig. The existing tests could not catch this because they build a synthetic config already in DB shape, so a client/server mismatch is invisible to them. The new tests push flags through the real config-mapper in both directions and assert the derived flags equal what mapUiToDbConfig would persist, so they fail if that mapping changes again. One test pins the root cause directly: that giteaConfig in the API payload carries no flags. * fix(ui): surface the mirror-options filter on desktop, and on organizations The overrides filter only existed inside the mobile filter drawer, so on desktop the Repositories page showed a "Custom" badge on overridden rows with no way to filter to them. Being able to answer "which repos deviate from my defaults" on this page is the reason the Configuration-page listing was dropped, so desktop could not do the one job it was given. Adds the control to the desktop filter row next to status and sort, using the bare Select-with-placeholder style those use rather than the drawer's labelled markup. The mobile control is unchanged. The Organizations page had a wider version of the same gap: it renders the "Custom options" badge but never had this filter on either layout, and OrganizationsList did not filter on it at all. Added the predicate plus the control in both its layouts, so the two pages behave the same. Also folds hasOverrides into activeFilterCount on both pages and into the Organizations clear-all reset. Without that the mobile filter badge undercounted an active overrides filter, and clearing filters left it set. * fix(ui): stop double-applying the mirrorMetadata switch when reading config mirrorOptionsToFlags ANDed each metadata component with mirrorMetadata on the way in. That is a write-path rule: mapUiToDbConfig already applies it when persisting, so a config saved through the settings UI has it baked into the stored flag. Applying it again on read double-applies it. The read path does not need it. mapDbToUiConfig puts the raw stored values into metadataComponents, so metadataComponents.issues is exactly giteaConfig.mirrorIssues, which is the field gitea.ts and gitea-enhanced.ts read. Mapping the components straight through is 1:1 with runtime behavior. Double-applying was invisible while the stored state was self-consistent, since false && false is still false. It diverged when the state did not come from the UI write path, which env vars allow: MIRROR_METADATA=false with MIRROR_ISSUES=true stores mirrorMetadata:false, mirrorIssues:true. The runtime mirrors issues; the dialog reported off. Measured against a config in that state, five flags misreported. I previously described this as an unrecoverable loss in the API shape. That was wrong. Both values reach the client separately and unmerged, so the payload carries full information and the loss was one we introduced. Rewrites the test that asserted the derived flags match what mapUiToDbConfig would persist. Comparing against the write derivation is what pinned the bug in place. It now asserts the property that matters, that flags derived from the API payload equal the stored flags the runtime reads, plus a case checking agreement with resolveMirrorOptions on an inconsistent config. Both fail if the AND returns.
Scripts Directory
This folder contains utility scripts for database management, event management, Docker builds, and LXC container deployment.
Database Management
Database Management Tool (manage-db.ts)
This is a consolidated database management tool that handles all database-related operations. It combines the functionality of the previous separate scripts into a single, more intelligent script that can check, fix, and initialize the database as needed.
Features
- Check Mode: Validates the existence and integrity of the database
- Init Mode: Creates the database only if it doesn't already exist
- Fix Mode: Corrects database file location issues
- Reset Users Mode: Removes all users and their data
- Auto Mode: Automatically checks, fixes, and initializes the database if needed
Running the Database Management Tool
You can execute the database management tool using your package manager with various commands:
# Checks database status (default action if no command is specified)
bun run manage-db
# Check database status
bun run check-db
# Initialize the database (only if it doesn't exist)
bun run init-db
# Fix database location issues
bun run fix-db
# Automatic check, fix, and initialize if needed
bun run db-auto
# Reset all users (for testing signup flow)
bun run reset-users
# Remove database files completely
bun run cleanup-db
# Complete setup (install dependencies and initialize database)
bun run setup
# Start development server with a fresh database
bun run dev:clean
# Start production server with a fresh database
bun run start:fresh
Database File Location
The database file should be located in the ./data/gitea-mirror.db directory. If the file is found in the root directory, the fix mode will move it to the correct location.
Event Management
The following scripts help manage events in the SQLite database:
Note
: For a more user-friendly approach, you can use the cleanup button in the Activity Log page of the web interface to delete all activities with a single click.
Remove Duplicate Events (remove-duplicate-events.ts)
Specifically removes duplicate events based on deduplication keys without affecting old events.
# Remove duplicate events for all users
bun scripts/remove-duplicate-events.ts
# Remove duplicate events for a specific user
bun scripts/remove-duplicate-events.ts <userId>
Fix Interrupted Jobs (fix-interrupted-jobs.ts)
Fixes interrupted jobs that might be preventing cleanup by marking them as failed.
# Fix all interrupted jobs
bun scripts/fix-interrupted-jobs.ts
# Fix interrupted jobs for a specific user
bun scripts/fix-interrupted-jobs.ts <userId>
Use this script if you're having trouble cleaning up activities due to "interrupted" jobs that won't delete.
Startup Recovery (startup-recovery.ts)
Runs job recovery during application startup to handle any interrupted jobs from previous runs.
# Run startup recovery (normal mode)
bun scripts/startup-recovery.ts
# Force recovery even if recent attempt was made
bun scripts/startup-recovery.ts --force
# Set custom timeout (default: 30000ms)
bun scripts/startup-recovery.ts --timeout=60000
# Using npm scripts
bun run startup-recovery
bun run startup-recovery-force
This script is automatically run by the Docker entrypoint during container startup. It ensures that any jobs interrupted by container restarts or application crashes are properly recovered or marked as failed.
Deployment Scripts
Docker Deployment
- build-docker.sh: Builds the Docker image for the application
- docker-diagnostics.sh: Provides diagnostic information for Docker deployments
LXC Container Deployment
Two deployment options are available for LXC containers:
-
Proxmox VE (online): Using the community-maintained script by Tobias (CrazyWolf13)
- Author: Tobias (CrazyWolf13)
- Available at: community-scripts/ProxmoxVED
- Pulls everything from GitHub
- Creates a privileged container with the application
- Sets up systemd service
-
gitea-mirror-lxc-local.sh: For offline/LAN-only deployment on a developer laptop
- Pushes your local checkout + Bun ZIP to the container
- Useful for testing without internet access
For detailed instructions on LXC deployment, see README-lxc.md.