This fixes issues #144, #577 and #320.
This update introduces a new options called "symlink-policy" that can be set to "store", "ignore" or "follow".
The default setting "store" means all symlinks are recorded as symlinks and recreated as symlinks on restore. This is chosen as default because it is the best replication of the actual disk contents.
Setting it to "ignore" simply means that no information is recorded about any symlinks.
And lastly, setting it to "follow" reverts to the previous default of treating symlinks as if they are normal files/folders.
Caveat: On Windows, you need Administrator privileges to create symlinks (when restoring), but not for reading them (on backup).
Since this change meant that the file attributes must be read, I also added support for the option "exclude-files-attributes", that can be used to ignore/exclude files with certain attributes, such as "hidden", "archive" or "encrypted".
Since I had to much about so much with getting the symlinks supported on Windows, I also added some extra logic to the Windows file handler, so it can now handle files and folders with up to 32k chars in the path. The logic is done so that the normal System.IO operations are used for paths with less than 260 characters (MAX_PATH), and on error or longer paths, the Win32 Unicode API is invoked. This should give maximal backwards compatibility and still allow backup and restore of files with long paths.
This commit also includes a BlockingQueue implementation that is supposed to make file/folder enumeration faster, but this has not been implemented because Duplicati depends on having a "whole world" view of the files and folders before running the backup (i.e. deeper changes required for this).
git-svn-id: https://duplicati.googlecode.com/svn/trunk@1381 59da171f-624f-0410-aa54-27559c288bec
The code should now "gracefully" handle missing timestamps.
For files with no modification time, they are assigned 1. jan. 1970 00:00:00 (EPOCH).
This has the drawback that files are read fully because we cannot rely on the modification timestamp, so it may be a bit slower.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@802 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
The implementation is a mix of the the three mentioned methods (issue #247).
There is a commandline option called --open-file-policy, which can be set to
"ignore", "snapshot" or "copy".
The "ignore" setting does the same as previous version, simply exclude the file from the
backup set.
If the setting is either "snapshot" or "copy", and the file it locked, it is opened in non-excusive read mode.
Once open, a in-memory signature file is generated. This is compared to the previous signature to detect changes,
just as the normal operation. If the file has not changed, the file is skipped as it would normally be.
If the file has changed and the policy is "snapshot", the file will be processed as normal,
but during the file read, a new signature is generated. This new signature is stored
in the signature archives. When writing the new signature, it is compared to the original signature.
If the file has changed during the backup operation, a warning is written to the output log.
If the policy was set to "copy", the modified file is copied to a temporary location,
while a new signature is generated. If the copy fails due to a file change, the file
is omitted from the backup set. If the copy succeeds, the file is known to not have changed
during the first access and the copy, which gives some assurance that the file is not
actively being written, at the expense of a copy operation.
The default setting is "snapshot", because this is likely to work in most scenarios,
and potentially produces a broken file rather than guarantee a missing file (by "ignore").
Note that when reading an open file, there is always a chance that the file is being written,
so you may end up with a broken file in the backup. This is also true for the VSS and LVM snapshots,
except for VSS-aware applications.
The "copy" method is meant to minimize this risk, but there is no guarantee that the file is in an
acceptable state, just because it has not been written for any period.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@588 59da171f-624f-0410-aa54-27559c288bec
Status: Fixed
USN is now activated by default on windows, but requires Administrative privileges.
If it fails to activate, it is silently ignored, and normal filesystem enumeration is used.
Set --usn-policy to:
"on" to get a warning if it fails,
"required" to abort the backup if it fails
"off" to disable USN
"auto" to silently ignore a fail
git-svn-id: https://duplicati.googlecode.com/svn/trunk@584 59da171f-624f-0410-aa54-27559c288bec
in the local format (eg. using backslash). This could
create trouble if the backup was made on windows and restored
on linux.
Update issue #229
Status: Fixed
I have updated the code to store the correct time for the
files, as well as the folders. The folder timestamp
adds a slight overhead to the volumes, as Zip
archives have no folder entries, so the timestamps
for folders are kept in a separate file.
The time stored in the existing archives is the date the backup
was made, not the last modified time, so it is not possible to
restore the modification date for existing archives.
When restoring new archives, the correct date is set
for both files and folders.
When restoring older archives, the backup date is used
for files, and nothing is done for folders.
I also changed the internal storage to use UTC time format
for all timestamps to avoid problems with backup/restores on
machines with different timezones.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@484 59da171f-624f-0410-aa54-27559c288bec
Added snapshot support using either VSS (win xp/vista/7) or LVM (linux).
Using the snapshot feature requires administrative/root privileges, and is toggled by the "snapshot-policy" option (by commandline or in advanced settings).
Also improved the validation of options to not show incorrect warnings.
This fixes issue #233.
git-svn-id: https://duplicati.googlecode.com/svn/trunk@461 59da171f-624f-0410-aa54-27559c288bec
kenneth@hexad.dk