This PR fixes a memory usage issue with MacOS Photos integration.
Prior to this fix, Duplicati would allow the Photos library to stream an unlimited amount of data. For large files this would require all resources to be kept fully in memory.
With this fix, the reader is limited to 8MB read-ahead, such that the maximum size of memory use is bounded by 8MBxReaders, where readers are default cores/2.
Additionally, this significantly reduces the memory allocation/deallocation in the C# layer when processing PhotoKit entries.
This PR adds detection of the MacOS Photos folder, and intercepts reads and replaces them with PhotoKit calls.
With this, it is possible to make backups of all MacOS Photos, even if they are not stored locally.
The previous versions would just make a backup of the on-disk structure, which was not guaranteed to contain all photos, but instead has various indexing for finding photos, and may contain some original photos.
The option `--photos-handling` controls how Duplicati now deals with the Photos folder. The options are:
- `LibraryOnly`: Same as before, just treat it as a folder
- `PhotosOnly`: Ignore the folder contents and just back up the actual photos
- `PhotosAndLibrary` (default): Make a backup of the photos and the library on-disk. This may cause images to be stored twice, but de-duplication will usually limit the storage increase.
The option `--photos-library-path` can be used to point to the on-disk Photo library that should be handled, in case the auto-detection does not pick it up. If this does not point to a valid Photoslibrary, or the path is not being backed up, no special handling will be done.
Note that the restore is not restoring into Photos itself, but instead restores into a sub-folder in the Photolibrary that is called `dup_backup`. To get the photos out after a restore, one needs to right-click the Photolibrary folder, and choose "Show package contents" and then the `dup_backup` folder is revealed.
This is done to keep all photos in the same folder, but avoid messing with the structure of the on-disk Photolibrary.
A future update could allow restoring back into Photos, and metadata is captured for each image to eventually allow this.
This fixes#6381
Kenneth Skovhede