The number of methods required by the ICompression interface is now reduce and all handling of path conversion and text encoding is now done in the CompressionWrapper class. This should make it easier to implement a new compression module, and reduce the number of errors that occur because of filename/encoding issues.
This commit also contains a fix for reporting a sane error message if the compression module is not found/loaded.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@1417 59da171f-624f-0410-aa54-27559c288bec
This fixes issues #144, #577 and #320.
This update introduces a new options called "symlink-policy" that can be set to "store", "ignore" or "follow".
The default setting "store" means all symlinks are recorded as symlinks and recreated as symlinks on restore. This is chosen as default because it is the best replication of the actual disk contents.
Setting it to "ignore" simply means that no information is recorded about any symlinks.
And lastly, setting it to "follow" reverts to the previous default of treating symlinks as if they are normal files/folders.
Caveat: On Windows, you need Administrator privileges to create symlinks (when restoring), but not for reading them (on backup).
Since this change meant that the file attributes must be read, I also added support for the option "exclude-files-attributes", that can be used to ignore/exclude files with certain attributes, such as "hidden", "archive" or "encrypted".
Since I had to much about so much with getting the symlinks supported on Windows, I also added some extra logic to the Windows file handler, so it can now handle files and folders with up to 32k chars in the path. The logic is done so that the normal System.IO operations are used for paths with less than 260 characters (MAX_PATH), and on error or longer paths, the Win32 Unicode API is invoked. This should give maximal backwards compatibility and still allow backup and restore of files with long paths.
This commit also includes a BlockingQueue implementation that is supposed to make file/folder enumeration faster, but this has not been implemented because Duplicati depends on having a "whole world" view of the files and folders before running the backup (i.e. deeper changes required for this).
git-svn-id: https://duplicati.googlecode.com/svn/trunk@1381 59da171f-624f-0410-aa54-27559c288bec
Update issue #296
I found and fixed a situation where a single file could be left open if the backup was stopped/aborted.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@1173 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
Duplicati now defaults to not upload a backup if nothing has changed.
This is indicated by a light-green check mark in the status window.
The option --upload-unchanged-backups can be set to revert to the previous way of working.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@958 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
Found and fixed a "slash vs. backslash" problem with files that span multiple volumes.
This will be included in the next preview release.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@950 59da171f-624f-0410-aa54-27559c288bec
The code should now "gracefully" handle missing timestamps.
For files with no modification time, they are assigned 1. jan. 1970 00:00:00 (EPOCH).
This has the drawback that files are read fully because we cannot rely on the modification timestamp, so it may be a bit slower.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@802 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
The folder list is now case-insensitive on Windows,
so the problem no longer arises.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@750 59da171f-624f-0410-aa54-27559c288bec
This commit includes an option to create an unencrypted verification file.
There is now a check to see if all volumes mentioned in the manifest exists,
before performing backup/restore operations.
This check is also performed on cleanup.
There is now a check that prevents auto-cleanup from removing more than
2 orphan files.
If a delete-older-than or delete-all-but-n-full is interrupted, the leftover
files will not be auto-deleted but must be deleted with the cleanup command,
which allows the user to see what files are delete.
Update issue #360
Status: Fixed
There is now a complete verification chain, where the manifest hashes are verified backwards.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@711 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
I found and fixed the issue, I was using an int instead of a long for the filesize,
which caused it to truncate.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@683 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
The implementation is a mix of the the three mentioned methods (issue #247).
There is a commandline option called --open-file-policy, which can be set to
"ignore", "snapshot" or "copy".
The "ignore" setting does the same as previous version, simply exclude the file from the
backup set.
If the setting is either "snapshot" or "copy", and the file it locked, it is opened in non-excusive read mode.
Once open, a in-memory signature file is generated. This is compared to the previous signature to detect changes,
just as the normal operation. If the file has not changed, the file is skipped as it would normally be.
If the file has changed and the policy is "snapshot", the file will be processed as normal,
but during the file read, a new signature is generated. This new signature is stored
in the signature archives. When writing the new signature, it is compared to the original signature.
If the file has changed during the backup operation, a warning is written to the output log.
If the policy was set to "copy", the modified file is copied to a temporary location,
while a new signature is generated. If the copy fails due to a file change, the file
is omitted from the backup set. If the copy succeeds, the file is known to not have changed
during the first access and the copy, which gives some assurance that the file is not
actively being written, at the expense of a copy operation.
The default setting is "snapshot", because this is likely to work in most scenarios,
and potentially produces a broken file rather than guarantee a missing file (by "ignore").
Note that when reading an open file, there is always a chance that the file is being written,
so you may end up with a broken file in the backup. This is also true for the VSS and LVM snapshots,
except for VSS-aware applications.
The "copy" method is meant to minimize this risk, but there is no guarantee that the file is in an
acceptable state, just because it has not been written for any period.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@588 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
USN is now activated by default on windows, but requires Administrative privileges.
If it fails to activate, it is silently ignored, and normal filesystem enumeration is used.
Set --usn-policy to:
"on" to get a warning if it fails,
"required" to abort the backup if it fails
"off" to disable USN
"auto" to silently ignore a fail
git-svn-id: https://duplicati.googlecode.com/svn/trunk@584 59da171f-624f-0410-aa54-27559c288bec
Added the --vss-exclude-writers option.
Update issue #260
Status: Fixed
I have now added a --vss-exclude-writers commandline option.
I had another look at the other supported options, and I consider this the most important.
I made the simplest possible version, so it only supports class guids, not component names
or writer instance ids.
If you need any of the other switches or the component names, please let me know.
I would also be interested in hearing that it actually works (I get no errors, but thats no proof).
You can specify multiple writers with a semicolon: --vss-exclude-writers="{d61d61c8-d73a-4eee-8cdd-f6f9786b7124};{0bada1de-01a9-4625-8278-69e735f39dd2}"
If you need the guids, you can get them with the following command in a command prompt:
vssadmin list writers
git-svn-id: https://duplicati.googlecode.com/svn/trunk@574 59da171f-624f-0410-aa54-27559c288bec
After a fix with source folder filters, the internal file "added_folders.txt"
contains the root folder entry, but not suffixed with a slash/backslash,
which caused an "index of of bounds" error when attempting to restore.
Update issue #295
Status: Fixed
I found 3 situations where an opened file was not closed.
Strange thing is that it should have been that way since the multivolume stuff was introduced.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@569 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
Added a find-last-version command to the commandline version,
it takes the same input as a restore operation, so
use --file-to-restore to supply a list of files to search for.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@559 59da171f-624f-0410-aa54-27559c288bec
Since encryption (and compression) is detected after being encountered, the encryption module selection has been removed from the user interface when restoring.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@527 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
A restore now defaults to background and some filenames are also shown while restoring.
The "direct restore" option cannot run in the background as it requires the schedule to be stored in the database.
This can be fixed after version 2.0
git-svn-id: https://duplicati.googlecode.com/svn/trunk@525 59da171f-624f-0410-aa54-27559c288bec
Fixed a protential race condition with warning messages and file counts (cosmetic only) when using async transfers.
Spelling correction, Seperator -> Separator.
Changed so a restore operation can also supply advanced options.
Changed the format of timestamps in filenames to follow the Duplicity convention, which removes the need for --time-separator and --use-short-filenames.
Moved the URL utility into Core, to avoid compilation errors with Monodevelop.
Reorganized the runner to better support the LiveControls on all operations rather than only for backup/restore.
Update issue #210
Status: Fixed
I have now changed the time format to follow the Duplicity naming convention.
This means that all filenames are now generated with the time set as "yyyyMMddTHHmmssZ".
This format is more portable across all platforms, and easily sortable/readable.
The drawback is that the Z indicates UTC time, so if one looks at the filenames,
the times are not local. The motivation for this is to avoid trouble
with daylight savings time offsets.
There is full backwards compatibility as long as backups have not used more than
one character for --time-separator (only possible via commandline).
This new format means that the options --time-separator and --use-short-filenames
are now deprecated and removed from the user interface. It is possible to
access them from the advanced options page, along with the --use-old-filenames option.
If anyone needs these options for some reason, let me know, otherwise they will be removed
in a later version.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@523 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
I have now added an option called --exclude-empty-folders
It excludes folders AFTER the filter has been applied, so if the folder is left empty due to the filter, the folder is not included either.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@520 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
It turns out that the fix to enable filters disabled the --file-to-restore option.
I have fixed that and also updated the restore output to correctly reflect what has happened during the restore.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@505 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
The backup can now proceed with a broken signature file.
A restore operation is not affected by broken signature files either,
although the restore will include previously deleted files,
if the signatures are damaged.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@492 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
The code to support this was already in place, but did
not work because Application.Exit() had been invoked,
making all messageboxes return Cancel without showing.
I also improved the logic a bit so a cancel does
not invoke the cleanup afterwards, making the exit
procedure faster.
I also added checks for the stop command to the
code that fetches the signature files, to make
the application respond faster to the stop command.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@490 59da171f-624f-0410-aa54-27559c288bec
in the local format (eg. using backslash). This could
create trouble if the backup was made on windows and restored
on linux.
Update issue #229
Status: Fixed
I have updated the code to store the correct time for the
files, as well as the folders. The folder timestamp
adds a slight overhead to the volumes, as Zip
archives have no folder entries, so the timestamps
for folders are kept in a separate file.
The time stored in the existing archives is the date the backup
was made, not the last modified time, so it is not possible to
restore the modification date for existing archives.
When restoring new archives, the correct date is set
for both files and folders.
When restoring older archives, the backup date is used
for files, and nothing is done for folders.
I also changed the internal storage to use UTC time format
for all timestamps to avoid problems with backup/restores on
machines with different timezones.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@484 59da171f-624f-0410-aa54-27559c288bec
Added snapshot support using either VSS (win xp/vista/7) or LVM (linux).
Using the snapshot feature requires administrative/root privileges, and is toggled by the "snapshot-policy" option (by commandline or in advanced settings).
Also improved the validation of options to not show incorrect warnings.
This fixes issue #233.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@461 59da171f-624f-0410-aa54-27559c288bec
This was based on the idea that the interfaces should remain static.
I hope this change is enough to ensure stable interfaces until release 1.2.
Overview of changes:
Moved all interfaces into the same dll.
Encryption and compression is now plugable modules, just as the backends.
Encryption/compression can now register an UI.
Encryption now uses AESCrypt as a default.
GPG does not default to using the --armor option.
Added support for generic modules, but none are written yet.
Added support for plugable settings pages in the "Options" dialog.
Fixed issue #148.
Duplicati now uses AESCrypt as the default encryption format.
Fixed issue #199.
GPG now supports custom commandline options.
Fixed issue #207.
Encryption modules are now plugable.
Fixed issue #118.
S3 credentials are now stored.
Fixed issue #151.
Backends can now register system wide options.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@427 59da171f-624f-0410-aa54-27559c288bec
