|
From: | Kenneth Loafman |
Subject: | Re: [Duplicity-talk] Restoring from PCA Backend - download preparation |
Date: | Sun, 27 Jun 2021 11:45:38 -0500 |
_______________________________________________Hi,
I’m using duplicity with multibackend (public cloud archive and object storage from ovh), as described in this documentation from ovh :
https://docs.ovh.com/gb/en/storage/pca/duplicity/
I’ve worked on a modification to make PCA Backend work better (for my usage at least !) : allow to unseal all volumes at once when restoring content from PCA, wait for unseal to complete for all volumes before launching download. I’ve used API documentation as mentioned here :
https://docs.ovh.com/gb/en/storage/pca/api/
This is a similar change to what has been implemented some time ago for Glacier cold storage :
https://lists.nongnu.org/archive/html/duplicity-talk/2020-07/msg00021.html
I’ve implemented the following things available in this commit : https://gitlab.com/erwanBoka/duplicity/-/commit/822e42ba3c26a5b32377e238a9b6040b9b087052
- Implement in BackendWrapper and MultiBackend pre_process_download (and redirect to sub-backends)
- Implement for pcabackend pre_process_download with ovh API as a blocking call, waiting for all volumes to be unsealed.
However, I’m a bit confused by what was implemented then (new hook pre_process_download_batch), compared to existing one pre_process_download.
- pre_process_download is called from dup_main/restore_get_patched_rop_iter with all filenames as parameters
- it’s implemented only once in _boto_single, but with a single filename as argument…
- pre_process_download_batch is essentially doing the same thing (call with all volume names)
I’m afraid my modifications in BackendWrapper and MultiBackend might cause pre_process_download to be called for _boto_single, and break things…
I haven’t modified anything yet in _boto_single, but my proposal would be :
- change pre_process_download method name in _boto_single to an « internal » method used only in _get
- keep only pre_process_download hook in dup_main
- implement it the way pre_process_download_batch is implemented today in _boto_single
I don’t have s3 access, and cannot test it though.
Let me know your view about this, and the path forward please !
Best regards,
Erwan
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[Prev in Thread] | Current Thread | [Next in Thread] |