haeaeae/duplicati
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Optimize synthetic filelist creation

Browse Source 
This PR adds a test that generates a synthentic filelist of a variable size, and then measures the time it takes to recreate the fileset.

As there was a reported slowdown, analysis indicated that the query would grow in the order of O=N^2, where N is the number of paths.

The code is then updated to instead use some temporary on-disk space to lay out the paths and then insert them as a new fileset. This should bring the time down to O=N.

Based on a test with 10k files, the query speed up is ~900x, but will be more on larger filesets.
This commit is contained in:
Kenneth Skovhede
2026-05-06 15:16:10 +02:00
parent 4244e71fee
commit ab487bdde2
3 changed files with 520 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/tests.yml
+1 -1
View File
@@ -35,7 +35,7 @@ jobs:
- name: Run unit tests with coverage
run: |
mkdir -p "$GITHUB_WORKSPACE/TestResults/unit"
dotnet test --no-build --verbosity minimal --filter "Category!=Integration&Category!=DiskImageLocal&ClassName!~SecretProviderSetSecretTests" --collect:"XPlat Code Coverage" --results-directory "$GITHUB_WORKSPACE/TestResults/unit" Duplicati.slnx
dotnet test --no-build --verbosity minimal --filter "Category!=Integration&Category!=DiskImageLocal&Category!=ExcludedFromCLI&ClassName!~SecretProviderSetSecretTests" --collect:"XPlat Code Coverage" --results-directory "$GITHUB_WORKSPACE/TestResults/unit" Duplicati.slnx
- name: Upload coverage reports to Codecov
if: always()