Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DPE-3684][DPE-6020] Reinitialise raft #611

Merged
merged 113 commits into from
Feb 4, 2025
Merged

Conversation

dragomirp
Copy link
Contributor

@dragomirp dragomirp commented Sep 3, 2024

Syncobj RAFT implementation, used as a standalone DCS for Patroni, cannot elect a leader if the cluster loses quorum and becomes read only. This will prevent Patroni from automatically switching over, even in cases where sync_standbys are available in the cluster and could take over as primary.

The PR adds logic to detect when the RAFT cluster becomes read only and updates promote-to-primary action to be able to trigger a RAFT re-initialisation.

Contains changes from #663

Copy link

codecov bot commented Sep 3, 2024

Codecov Report

Attention: Patch coverage is 76.25571% with 52 lines in your changes missing coverage. Please review.

Project coverage is 71.85%. Comparing base (58449da) to head (bf025dc).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/charm.py 76.76% 19 Missing and 14 partials ⚠️
src/cluster.py 75.00% 17 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #611      +/-   ##
==========================================
+ Coverage   71.38%   71.85%   +0.47%     
==========================================
  Files          15       15              
  Lines        3253     3464     +211     
  Branches      480      535      +55     
==========================================
+ Hits         2322     2489     +167     
- Misses        812      844      +32     
- Partials      119      131      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dragomirp dragomirp force-pushed the dpe-3684-reinitialise-raft branch from 4c7e201 to 29aaf18 Compare September 3, 2024 19:36
@dragomirp dragomirp force-pushed the dpe-3684-reinitialise-raft branch from e69800c to adbf7eb Compare September 10, 2024 23:56
commit 72ddcac
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 29 16:31:18 2025 +0200

    Timeout check

commit adb8ba9
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 29 15:08:52 2025 +0200

    Missed auto recovery

commit 7d2070a
Merge: d084026 16e36d0
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 29 11:16:53 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit d084026
Merge: 9c00278 5d49c72
Author: Dragomir Penev <[email protected]>
Date:   Mon Jan 27 16:03:19 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 9c00278
Author: Dragomir Penev <[email protected]>
Date:   Thu Jan 23 23:24:39 2025 +0200

    Remove dead code

commit 84c6601
Merge: caf076f 7d49b47
Author: Dragomir Penev <[email protected]>
Date:   Thu Jan 23 18:30:01 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit caf076f
Author: Dragomir Penev <[email protected]>
Date:   Thu Jan 23 18:02:12 2025 +0200

    Correct timeout exception

commit ecb70d5
Author: Dragomir Penev <[email protected]>
Date:   Thu Jan 23 16:18:45 2025 +0200

    Check for Patroni self-healing

commit b3f44bc
Merge: 7d4bb84 3681434
Author: Dragomir Penev <[email protected]>
Date:   Thu Jan 23 16:04:49 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 7d4bb84
Merge: f7cef49 de52122
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 22 16:57:45 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit f7cef49
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 22 16:52:44 2025 +0200

    Unit tests

commit e1de0c4
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 22 16:22:19 2025 +0200

    Remove replica test

commit b76412e
Merge: 5f6ad5e 6ae7e52
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 22 16:19:02 2025 +0200

    Merge branch 'main' into dpe-3684-three-units

commit 5f6ad5e
Author: Dragomir Penev <[email protected]>
Date:   Tue Jan 21 22:32:39 2025 +0200

    Switch test order

commit a39bae9
Author: Dragomir Penev <[email protected]>
Date:   Tue Jan 21 17:48:55 2025 +0200

    Regular promote

commit 66088cb
Merge: 720acf0 6917be3
Author: Dragomir Penev <[email protected]>
Date:   Mon Jan 20 01:47:28 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 720acf0
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 8 16:55:48 2025 +0200

    Use a sync unit to run the restore action

commit 9b3bcfd
Merge: 86d14b9 6ffaf17
Author: Dragomir Penev <[email protected]>
Date:   Wed Jan 8 15:49:31 2025 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 86d14b9
Author: Dragomir Penev <[email protected]>
Date:   Mon Jan 6 10:55:02 2025 +0200

    GH build job workaround

commit c062fc0
Author: Dragomir Penev <[email protected]>
Date:   Sun Jan 5 00:42:32 2025 +0200

    Fix tests

commit df65bd0
Author: Dragomir Penev <[email protected]>
Date:   Mon Dec 30 22:22:13 2024 +0200

    Wrong call

commit 5f3609f
Author: Dragomir Penev <[email protected]>
Date:   Mon Dec 30 20:04:23 2024 +0200

    Raise flag on read only primary

commit e459d05
Author: Dragomir Penev <[email protected]>
Date:   Thu Dec 26 16:23:45 2024 +0200

    Tests

commit ffd6d58
Author: Dragomir Penev <[email protected]>
Date:   Tue Dec 24 17:41:26 2024 +0200

    Bump timeout

commit 74b4d48
Author: Dragomir Penev <[email protected]>
Date:   Tue Dec 24 17:01:35 2024 +0200

    Switch action

commit dd88198
Author: Dragomir Penev <[email protected]>
Date:   Tue Dec 24 16:22:34 2024 +0200

    Promote via existing action

commit 55a7a7e
Merge: 2308e8c a20cf72
Author: Dragomir Penev <[email protected]>
Date:   Tue Dec 24 15:51:57 2024 +0200

    Merge branch 'dpe-3684-three-units' into dpe-3684-reinit-action

commit a20cf72
Merge: 1e17ced d155abf
Author: Dragomir Penev <[email protected]>
Date:   Tue Dec 24 15:51:23 2024 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 2308e8c
Author: Dragomir Penev <[email protected]>
Date:   Tue Nov 26 19:21:16 2024 +0200

    Run reinit as part of the action for leader

commit e224caf
Author: Dragomir Penev <[email protected]>
Date:   Tue Nov 26 18:13:13 2024 +0200

    Add experimental action

commit 1e17ced
Author: Dragomir Penev <[email protected]>
Date:   Tue Nov 26 16:31:08 2024 +0200

    Wrong parameter

commit 5e18530
Author: Dragomir Penev <[email protected]>
Date:   Tue Nov 26 16:28:14 2024 +0200

    Try to reinit on update status

commit 07fd969
Merge: ebf1361 a307bd8
Author: Dragomir Penev <[email protected]>
Date:   Mon Nov 25 18:59:58 2024 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit ebf1361
Merge: 782979b 5535896
Author: Dragomir Penev <[email protected]>
Date:   Fri Nov 22 16:22:16 2024 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 782979b
Merge: da33c75 0896245
Author: Dragomir Penev <[email protected]>
Date:   Tue Nov 19 12:50:18 2024 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit da33c75
Author: Dragomir Penev <[email protected]>
Date:   Wed Oct 30 00:05:04 2024 +0200

    Linting

commit ae5b560
Author: Dragomir Penev <[email protected]>
Date:   Tue Oct 29 23:54:44 2024 +0200

    Degraded status message

commit cc16d5f
Author: Dragomir Penev <[email protected]>
Date:   Mon Oct 28 22:25:12 2024 +0200

    Set maintanance status during reinit

commit 5088114
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 23:11:21 2024 +0200

    Unit tests

commit 634a9bb
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 22:58:32 2024 +0200

    Track candidate by cluster status

commit c0fece5
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 21:40:41 2024 +0200

    Log roles removed

commit e06c077
Merge: 7123659 c769b8e
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 20:27:38 2024 +0200

    Merge branch 'dpe-3684-reinitialise-raft' into dpe-3684-three-units

commit 7123659
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 19:45:42 2024 +0200

    Parameterised test

commit 5d972ac
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 18:37:41 2024 +0200

    Parameterised test

commit 42899c1
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 15:13:29 2024 +0200

    Pass the machine name when stopping

commit e0a09ae
Author: Dragomir Penev <[email protected]>
Date:   Sun Oct 27 13:46:39 2024 +0200

    Three units scaling tests
@dragomirp dragomirp changed the title [DPE-3684] Reinitialise raft [DPE-3684][DPE-6020] Reinitialise raft Feb 4, 2025
@dragomirp dragomirp merged commit fa7d3e4 into main Feb 4, 2025
97 checks passed
@dragomirp dragomirp deleted the dpe-3684-reinitialise-raft branch February 4, 2025 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants