 Carl Johnsen
|
6ba956dfcd
|
Moved all database operations to the LocalRestoreDatabase.
Also changed the old changes, so that the functions related to the legacy restore remains untouched.
|
2024-12-03 08:35:14 +01:00 |
|
|
Carl Johnsen
|
6c66fee921
|
Cleanup FileProcessor and rework docstrings
|
2024-12-02 20:27:21 +01:00 |
|
|
Carl Johnsen
|
b6ed72bd6c
|
Reworked the restore CSP network
|
2024-12-02 15:09:23 +01:00 |
|
|
Carl Johnsen
|
3156d43adc
|
BlockVolumeReaders are expensive to create - cache them instead.
|
2024-12-02 10:55:45 +01:00 |
|
|
Carl Johnsen
|
cdf0c28fd8
|
Added internal profiling as an option. Currently only implemented in the restore process.
|
2024-11-29 05:49:03 +01:00 |
|
|
Carl Johnsen
|
87747b5580
|
The metadata SQL query is now prepared in the fileprocessor
|
2024-11-29 05:46:17 +01:00 |
|
|
Carl Johnsen
|
6aa202fbea
|
Already present files were incorrectly truncated
|
2024-11-29 03:38:53 +01:00 |
|
|
Carl Johnsen
|
deb98e3322
|
CreateDirectoryStructure wouldn't always create the target folders, if the file database doesn't list the folder. Empty files should create the directory, if it doesn't exist.
|
2024-11-28 10:43:51 +01:00 |
|
|
Carl Johnsen
|
9bcda7d6cf
|
Symlinks are now handled correctly
|
2024-11-28 09:22:09 +01:00 |
|
|
Carl Johnsen
|
1ecb6e7c35
|
Fixed the metadata block query so that it finds the correct blocks
|
2024-11-27 11:28:27 +01:00 |
|
|
Carl Johnsen
|
4128ea3ff3
|
Empty files can also not have a data block associated
|
2024-11-27 11:27:55 +01:00 |
|
|
Carl Johnsen
|
7aa9db533e
|
Added preliminary metadata restoration
|
2024-11-26 20:01:19 +01:00 |
|
|
Carl Johnsen
|
164b2a212c
|
Already restored blocks are now tracked according to the cache tracking strategy as well
|
2024-11-25 17:32:56 +01:00 |
|
|
Carl Johnsen
|
823b896da1
|
Removed redundant TODOs
|
2024-11-25 15:23:59 +01:00 |
|
|
Carl Johnsen
|
7c79535572
|
VolumeDownloader now deletes temporary volumes from both cache and disk when they are no longer needed during restore.
|
2024-11-25 15:16:03 +01:00 |
|
|
Carl Johnsen
|
27c4f93af2
|
The FileProcessor bursts as much as possible by default
|
2024-11-25 08:53:27 +01:00 |
|
|
Carl Johnsen
|
0cd9374bed
|
Added comments and docstrings
|
2024-11-22 15:26:04 +01:00 |
|
|
Carl Johnsen
|
890fa2ce52
|
Add copyright notices
|
2024-11-22 11:47:07 +01:00 |
|
|
Carl Johnsen
|
7094786e23
|
Removed unnessecarily complex nested ternary statements.
|
2024-11-22 11:26:01 +01:00 |
|
|
Carl Johnsen
|
c798910658
|
If the target file already exists and is verified, do nothing more
|
2024-11-22 11:25:35 +01:00 |
|
|
Carl Johnsen
|
cacc5d18a0
|
Added tracking of errors of local files during restoration
|
2024-11-21 15:14:29 +01:00 |
|
|
Carl Johnsen
|
916d35e928
|
Added logic for utilizing the original data when restoring, if it matches the hashes in the database.
|
2024-11-21 10:39:48 +01:00 |
|
|
Carl Johnsen
|
b9a4b8c934
|
Moved local file verification to its own function
|
2024-11-21 07:19:13 +01:00 |
|
|
Carl Johnsen
|
d109bf5db1
|
The FileProcessor now verifies whether the target file has already been restored.
|
2024-11-20 16:33:33 +01:00 |
|
|
Carl Johnsen
|
9955647768
|
Changed the bursting strategy of the FileProcessor to fill up the pipeline, and then issue a new request after every response
|
2024-11-19 11:34:10 +01:00 |
|
|
Carl Johnsen
|
4d368f5a17
|
Changed the FileProcessor to use prepared statements
|
2024-11-19 10:33:50 +01:00 |
|
|
Carl Johnsen
|
75bda5f6c7
|
Added dryrun logic and logging to the restore processes
|
2024-11-15 19:02:30 +01:00 |
|
|
Carl Johnsen
|
339e06ce9c
|
Added logging and error handling to FileProcessor
|
2024-11-15 14:50:52 +01:00 |
|
|
Carl Johnsen
|
b7aa1a3856
|
Added on-the-fly file hash verification
|
2024-11-15 13:50:45 +01:00 |
|
|
Carl Johnsen
|
5fcdaf334a
|
FileProcessor now bursts their block requests
|
2024-11-15 13:40:21 +01:00 |
|
|
Carl Johnsen
|
dc73380fe8
|
To minimize database requests (or at least where they emerge), query the database once per block per file, once per newly downloaded volume, and once to build the file list. The results are then "shipped" around the network along with the previous messages.
|
2024-11-13 16:55:47 +01:00 |
|
|
Carl Johnsen
|
6f354e6d08
|
Added some progress tracking and removed some debug printing
|
2024-11-08 15:24:56 +01:00 |
|
|
Carl Johnsen
|
ec738f4063
|
Added handling of retired exception to close down the process network
|
2024-11-08 14:35:44 +01:00 |
|
|
Carl Johnsen
|
78aa077c2b
|
Handled empty files, where the block volume ID is -1
|
2024-11-08 13:26:29 +01:00 |
|
|
Carl Johnsen
|
997472127a
|
FileProcessor now writes to disk - the initial circle is complete!
|
2024-11-08 10:43:19 +01:00 |
|
|
Carl Johnsen
|
a6f6cb7f39
|
CoCoL did not like the array of ChannerMarkers
|
2024-11-08 08:45:00 +01:00 |
|
|
Carl Johnsen
|
b41b43f430
|
Now the volume ID is found alongside the block ID
|
2024-11-08 08:02:17 +01:00 |
|
|
Carl Johnsen
|
adf9d4acf4
|
Added the draft block manager
|
2024-11-08 07:20:18 +01:00 |
|
|
Carl Johnsen
|
50c6fb434c
|
Got the list of block IDs per file
|
2024-11-08 05:56:48 +01:00 |
|
|
Carl Johnsen
|
60fa5fe031
|
Started the draft FileProcessor process
|
2024-11-08 03:55:46 +01:00 |
|