This is inspired largely by the new (/ returning) feature of OneDrive in Windows 10 Fall Creator's Update, which downloads files on-demand.
A side effect of that change is that the OneDrive folder (and subfolders of it) are marked as reparse points, even though they are not technically standard symlinks.
With this change, a new extension method IsSymlink is added for ISnapshotService and ISystemIO, which checks both the file attributes (for reparse point) and the symlink target path (if null, the path is not treated as a symlink).
All places that previously checked only the file attributes have been updated to use either this method (or at least the same logic, in the case of the core BackupHandler file check).
One side effect of this change is that '--symlink-policy=store' no longer ignores empty symlinks - they are now treated as regular files.
If there are empty symlinks, they will now be backed up as if they were regular files, but I don't know what the conditions are that create symlinks like that, so this might not effect anything in practice.
This fixes issues #144, #577 and #320.
This update introduces a new options called "symlink-policy" that can be set to "store", "ignore" or "follow".
The default setting "store" means all symlinks are recorded as symlinks and recreated as symlinks on restore. This is chosen as default because it is the best replication of the actual disk contents.
Setting it to "ignore" simply means that no information is recorded about any symlinks.
And lastly, setting it to "follow" reverts to the previous default of treating symlinks as if they are normal files/folders.
Caveat: On Windows, you need Administrator privileges to create symlinks (when restoring), but not for reading them (on backup).
Since this change meant that the file attributes must be read, I also added support for the option "exclude-files-attributes", that can be used to ignore/exclude files with certain attributes, such as "hidden", "archive" or "encrypted".
Since I had to much about so much with getting the symlinks supported on Windows, I also added some extra logic to the Windows file handler, so it can now handle files and folders with up to 32k chars in the path. The logic is done so that the normal System.IO operations are used for paths with less than 260 characters (MAX_PATH), and on error or longer paths, the Win32 Unicode API is invoked. This should give maximal backwards compatibility and still allow backup and restore of files with long paths.
This commit also includes a BlockingQueue implementation that is supposed to make file/folder enumeration faster, but this has not been implemented because Duplicati depends on having a "whole world" view of the files and folders before running the backup (i.e. deeper changes required for this).
git-svn-id: https://duplicati.googlecode.com/svn/trunk@1381 59da171f-624f-0410-aa54-27559c288bec
Kenneth Skovhede