Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disaster Recovery process #469

Open
RaoulHC opened this issue Oct 25, 2023 · 0 comments
Open

Disaster Recovery process #469

RaoulHC opened this issue Oct 25, 2023 · 0 comments
Labels
reflexion More thought needed on this topic

Comments

@RaoulHC
Copy link
Contributor

RaoulHC commented Oct 25, 2023

Once we have #452 in we'll be storing both processed tarballs, and the original ones.

We should allow the original ones to be stored in a separate blob store so that they can be put in cold storage (we can reuse the blob store effect but use effectful's labelled effects to distinguish the two), and add a way of recovering from what's put in cold storage. Some thought might be warranted for some of the other metadata fields for releases, with #454 more fields will become easily recoverable but others would still be lost, without putting more data in cold storage.

Process as envisioned by @tchoutri is as follows:

  1. A tarball gets uploaded
  2. The original gets stored on disk, then sent to a cold storage.
  3. The processed version gets stored within Flora
  4. The processed version gets cached by varnish/nginx/squid/whatever
  5. a) We need to refine / fix a bug in the processing of the tarballs, which is good because we have the originals at hand, so we can re-import them.
    b) We need to recover from an incident: A backup of the tarballs can be re-imported from the cold storage.
@tchoutri tchoutri changed the title DR recovery process Disaster Recovery process Oct 25, 2023
@tchoutri tchoutri added the reflexion More thought needed on this topic label Oct 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reflexion More thought needed on this topic
Projects
None yet
Development

No branches or pull requests

2 participants