Skip to content

[DPE-6678] PostgreSQL Config Improvement #778

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Mar 22, 2025
Merged

Conversation

Pepsiqqq
Copy link
Contributor

This is copy of K8s PR, however there is an issue that PostgreSQL stucks in Awaiting for member to start. I think it's related to this issue because there is the same error (Failed to list PostgreSQL database users: could not translate host name "None" to address: Temporary failure in name resolution). On K8s side everything works as intended.

Added all missing parameters from PostgreSQL config analysis (Added params 25.05):
https://docs.google.com/spreadsheets/d/1CbJMmSSP5eyu08y5vM8RNGrjiSxOLsa7RelwwUsi-vg/edit?pli=1&gid=815242942#gid=815242942

Were not added:

  1. client_encoding: Session only parameter that can't be set by Patroni. Source
  2. transaction_deferrable: This parameter cannot be set in the configuration file, or from any source other than live SQL. Source
  3. transaction_isolation: This parameter cannot be set in the configuration file, or from any source other than live SQL. Source
  4. transaction_read_only: This parameter cannot be set in the configuration file, or from any source other than live SQL. Source

Notice:

  1. old_snapshot_threshold: Will be deleted in 17 version. Source
  2. default_table_access_method: Added dynamic validation.

Also sorted config to improve readability.

@dragomirp dragomirp requested review from a team, taurus-forever, dragomirp and marceloneppel and removed request for a team February 20, 2025 11:19
@dragomirp dragomirp added the enhancement New feature, UI change, or workload upgrade label Feb 21, 2025
@Pepsiqqq Pepsiqqq changed the title PostgreSQL Config Improvement [DPE-6678] PostgreSQL Config Improvement Feb 24, 2025
Copy link

codecov bot commented Feb 25, 2025

Codecov Report

Attention: Patch coverage is 75.11962% with 104 lines in your changes missing coverage. Please review.

Project coverage is 72.11%. Comparing base (dd27445) to head (1110909).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/config.py 75.48% 51 Missing and 51 partials ⚠️
src/charm.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #778      +/-   ##
==========================================
+ Coverage   71.97%   72.11%   +0.13%     
==========================================
  Files          15       15              
  Lines        3479     3862     +383     
  Branches      532      583      +51     
==========================================
+ Hits         2504     2785     +281     
- Misses        845      897      +52     
- Partials      130      180      +50     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

# Conflicts:
#	lib/charms/postgresql_k8s/v0/postgresql.py
@dragomirp
Copy link
Contributor

Hi, @Pepsiqqq, please sync up with main, it should skip the tests that cannot run on forks.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@marceloneppel
Copy link
Member

Hi, @Pepsiqqq! Thanks for syncing the branch. We're checking why a lot of tests are failing. We found that the replication HBA rules are not being filled correctly for some reason. Still investigating.

@marceloneppel
Copy link
Member

I'm not sure yet, but it seems something related to RAFT and the number of config options we try to set on PG:

2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]: Exception in thread Thread-2 (_autoTickThread):
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]: Traceback (most recent call last):
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self.run()
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3.10/threading.py", line 953, in run
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self._target(*self._args, **self._kwargs)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/patroni/dcs/raft.py", line 271, in _autoTickThread
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self.doTick(self.conf.autoTickPeriod)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/pysyncobj/syncobj.py", line 532, in doTick
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self._onTick(timeToWait)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/patroni/dcs/raft.py", line 261, in _onTick
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     super(KVStoreTTL, self)._onTick(timeToWait)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/patroni/dcs/raft.py", line 97, in _onTick
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     super(DynMemberSyncObj, self)._onTick(timeToWait)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/pysyncobj/syncobj.py", line 622, in _onTick
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self._checkCommandsToApply()
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/pysyncobj/syncobj.py", line 456, in _checkCommandsToApply
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self.__raftLog.add(command, idx, term)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/pysyncobj/journal.py", line 201, in add
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self.__journalFile.write(self.__currentOffset, cmdData)
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:   File "/usr/lib/python3/dist-packages/pysyncobj/journal.py", line 104, in write
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]:     self.__mm[offset:offset + size] = values
2025-03-19T20:44:22Z charmed-postgresql.patroni[4826]: IndexError: mmap slice assignment is wrong size

@marceloneppel marceloneppel mentioned this pull request Mar 20, 2025
@marceloneppel
Copy link
Member

Hi, @Pepsiqqq! Could you apply the 2 most recent commits from #802 in your PR? They should fix the above error that is happening in the integration tests.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was signed with the committer’s verified signature.
Pepsiqqq Danylo Bereznenko

Verified

This commit was signed with the committer’s verified signature.
Pepsiqqq Danylo Bereznenko
@Pepsiqqq
Copy link
Contributor Author

Thank you, seem like everything works. I deployed charm and tried to change config with juju config postgresql asd=asd , and it worked.

@marceloneppel
Copy link
Member

Thanks for the great contribution, @Pepsiqqq!

@marceloneppel marceloneppel merged commit 26a7982 into canonical:main Mar 22, 2025
93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature, UI change, or workload upgrade
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants