Null-merge branch '8.0' (after 8.0.37) into trunk (8.4) #5389
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Split router/cmake/configure.cmake to allow reuse. Change-Id: I4f4ca13729767e14ff75d0d915b124596ed1e132
index scan Description: - Indexes are ordered based on their keys. Loose index scan effectively jumps from one unique value (or set of values) to the next based on the index’s prefix keys. - To “jump” values in an index, we use the handler call: ha_index_read_map(). - the first range read sets an end-of-range value to indicate the end of the first range. - The next range read does not clear the previous end-of-range value and applies it to the current range. - Since the end-of-range value has already been crossed in the previous range read, this causes the reads to stop. So the iteration is finished with the current range without moving onto the next range(unique set of values)resulting in an incorrect query result. Fix: - In order to find the next unique value, the old end-of-range value is cleared. Change-Id: I84290fb794db13ec6f0795dd14a92cf85b9dad09
Problem: mysqldump not sanitizing the version string obtained from server which may lead to injecting malicious commands to the output. Fix: added function sanitizing the version string by cutting off illegal part and issuing warning. Test: check the server version in the output with and without injected payload. Change-Id: I1f19e1c90bdb8d444285e427092face3bb16da01
In jtie_gcalls.hpp, macros are used to create lists and templates. Many of these could stringify or concatenate up to 20 parameters, but capacity beyond about 13 is not needed. This patch shortens the lists. Clang will sometimes print a "rejecting template" error for each candidate template not chosen, so this helps to reduce the number of error messages generated before the compiler finally gives up. Change-Id: Iddbb25d0a6e5740815b686259c734a0bbf434d8d
An earlier version of jtie_gcalls generated two distinct templates for gcall_mfv() and gcall_mfr() to map the const and non-const variants of otherwise identical method calls. This strategy was abandoned because some very old C++ compilers could not handle the macro invocations that were used, and because Microsoft's compilers considered the two templates to be equivalent. Microsoft compilers still behave the same way, but Clang requires distinct templates. This patch restores the earlier design, but only in non-Microsoft environments. Change-Id: I6694c7cdc096a6675e71bcd843fec99421367c70
In jtie_tconv_object_impl.hpp, add a missing specialization of Target<J, C> for const C. Fix an incorrect mapping in myjapi_classes.hpp for C0::check(). Then undefine JTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION, and compile and run the tests (use "ctest -R jtie"). Change-Id: I5911bb6ef9ea149c2242378d39956a4d4afaab84
The jtie mapping for NdbOperation::getNdbErrorLine() should be marked as const-overloaded. The jtie mapping for Ndb_cluster_connection::set_service_uri() was declared as void, but should actually return int. The mappings for three const methods were defined to use the _t variant of the object trait type, but need to use the _ct variant. Change-Id: I4ab9e7a093c74edef72e2141d701e2e6c32b8bb6
In ndbjtie_defs.hpp, remove the "XXXXX temporary, for testing" comment from 2011, and do not define NDBJTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION or NDBJTIE_USE_WRAPPED_VARIANT_FOR_OVERLOADED_FUNCTION. This will enable use of the unwrapped mappings for most of the NDB API on all platforms. Change-Id: Id132500995cbaee62e7c1e5aa40f9272613501bb
Remove most JTie wrappers. They are no longer needed. The only remaining method wrappers are a small set for overloaded methods that have both const and non-const versions. Use "unifdef -UNDBJTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION" and "unifdef -UNDBJTIE_USE_WRAPPED_VARIANT_FOR_OVERLOADED_FUNCTION" to remove wrapped variants from the ndbapi and mysql_utils jtie header files. Change-Id: Ie08c2242aed2bfda02defe76fef40bd0bad6e9e7
In MySQL Cluster 8.0.31 several issues were introduced in ndb_redo_log_reader. Below fixes have been done: - For backward compatibility the legacy options with single minus for example -lap are now allowed after file argument. - One can now provide the mandatory numeric value for --mbyte/-mbyte, --page/-page, --pageindex/-pageindex options. - Option --mbyte now allows up to 1023 instead of 15 as value since we support fragment redo log file sizes up to 1GB. - When using --mbyte option it starts reading the file from given MB, previously it started from 4 times the given value. - Now it reads data only once, before it stepped back 3 quarter of a MB for every MB read and read it again. - Remove cast-qual warning in convert_legacy_options in reader.cpp. Added new test ndb.ndb_redo_log_reader. Change-Id: Iadfabcab9532eddf1314aeb618c72c31ab7b4a6f
…mented. ndb_redo_log_reader had several issues for example it failed on valid files with: Record type = 0 not implemented. Error in redoLogReader(). Exiting! And also warned about wrong checksums. The following changes were done: - Ignore unused (zero filled) pages. - Allow partial (header only) pages. - Treat checksum value 37 as no checksum disabled. Which is the case for release build. Checksum is only done in debug build. - Show mbyte up to 1023 rather than up to 15 in help, still no check. We support fragment log file size up to 1GB. - Spell FfragmentId as FragmentId. - Fix cast-qual warning when using PrepareOperationRecord::m_keyInfo. Change-Id: I40304555488fcfb1b3f0949096c61fdfff6171c0
…mented. ndb_redo_log_reader had several issues for example it failed on valid files with: Record type = 0 not implemented. Error in redoLogReader(). Exiting! And also warned about wrong checksums. The following changes were done: - Ignore unused (zero filled) pages. - Allow partial (header only) pages. - Treat checksum value 37 as no checksum disabled. Which is the case for release build. Checksum is only done in debug build. - Show mbyte up to 1023 rather than up to 15 in help, still no check. We support fragment log file size up to 1GB. - Spell FfragmentId as FragmentId. - Fix cast-qual warning when using PrepareOperationRecord::m_keyInfo. A new test ndb.ndb_redo_log_reader was added. Change-Id: I40304555488fcfb1b3f0949096c61fdfff6171c0
Change-Id: Icc7257a29ebcf899b6809238ca1bcb5ef8fc61fc
Change-Id: I246b76d3bb974734edd426223649e6f25eef5ffc
Support for encryption was added to ndb_redo_log_reader tool. Change-Id: If28dc314c2fab732474447db68e1a338248695e7
…nk Windows Problem: Slow Windows machines cause unexpected GR errors during tests: "There was an error when connecting to the donor server.." "For details please check.." Tests lacked these error checks, leading to false positives. Solution: Added suppressions for error messages in the test cases. Change-Id: I9d54f2a8190cdd61ecf56209ac567ceb765133ba
This commit backports the fix to 8.0 This patch will solve the following duplicates of this bug: Bug #112425: trx_t might be Use-After-Free in innobase_commit_by_xid Bug #99643: innobase_commit_by_xid/innobase_rollback_by_xid is not thread safe Bug #105036: trx would be used after free in `innobase_commit_by_xid` and rollback Background: TrxInInnoDB is a RAII wrapper for trx_t object used to track if the transaction's thread is currently executing within InnoDB code. It is acquired on all entry points, and as Innodb can be entered "recursively", the trx->in_depth is used to track the balance of enters and exits. On the outermost enter, the thread additionally checks if trx->in_innodb has the TRX_FORCE_ROLLBACK (0x8000 0000) flag set, which means a high priority transaction is attempting an asynchronous rollback of this transaction, so to avoid races, this thread should wait for the rollback to complete. Issue: TrxInInnoDB's destructor calls exit which resets in_depth and in_innodb increased by enter. However innobase_commit_by_xid and innobase_rollback_by_xid calls trx_free_for_background which returns the trx back to the pool, before the destructor is called. If this trx is being reused by another thread, it can lead to data-race and corrupted value of in_depth and in_innodb. If in_depth gets the value of -1, subsequent calls to enter and exit will bump in_innodb by one. This can lead to indefinite wait if in_innodb reaches TRX_FORCE_ROLLBACK. Fix: Ensure that TrxInInnoDB calls exit before returning the trx object to the pool. Further add checks to catch corrupt values of in_depth when freeing trx. Trx state validation before free was missed in trx_free_prepared_or_active_recovered Thanks to Shaohua Wang (Alibaba, Ex-Innodb) for the contribution Change-Id: Ibf79bec85ffa0eaf65f565c169db61536bff10a2
ndb.ndbinfo_plans result
Problem:
ndb.ndbinfo_plans have failed been seen to fail as below five times since
test part for WL#11968 was introduced January 2022.
Failures have mostly been on Windows but also on linux with ASAN.
CURRENT_TEST: ndb.ndbinfo_plans
--- .../mysql-test/suite/ndb/r/ndbinfo_plans.result
2024-02-11 06:00:21.000000000 +0300
+++ "...\log\ndbinfo_plans.reject"
2024-02-11 11:58:09.260695300 +0300
@@ -297,7 +297,6 @@
2
NDB$BLOBEVENT_REPL$mysql/ndb_schema_3
NDB$BLOB_4_3
3
REPL$mysql/ndb_schema_result
ndb_schema_result
4
ndb_index_stat_head_event
ndb_index_stat_head
-5
REPL$mysql/ndb_apply_status
ndb_apply_status
## Query uses primary keys on both tables:
EXPLAIN SELECT event_id, e.name, table_name FROM events e
Analysis:
Test assumes that event_id are constant but that can't be assumed, for
example when prior test has recreated tables or events.
Solution:
Rewrite to find list of event_ids by table name which are known to be
stable identifier.
Change-Id: I2ba83b0a715ea2ca2c5a75dba4e1e8ea635391c1
…t one redo file is missi
After calling rename(..) on POSIX platform one should fsync the
(destination) directory node to make sure change is persisted to disc.
Redo Log module was attempting fsync("data-dir/") instead of
fsync("data-dir/#innodb_redo/").
This could manifest as data loss and InnoDB not being able to start up,
if a power-outage (or OS crash) would happen right after rename, and
before the change to directory got persisted to disc on its own pace.
Similar problem could occur if a snapshot of the file system was taken
at that time.
The way this bug would manifest in practice is that when InnoDB needs a
new redo log file with number 280 (because 279th has become full of
data), what it does is the following (see log_files_produce_file()):
1. create "#ib_redo280_tmp" (using open(..) syscall)
2. resize "#ib_redo280_tmp" (using fallocate(..) + fsync/fdatasync())
3. close("#ib_redo280_tmp")
4. open("#ib_redo280_tmp")
5. write the correct headers to "#ib_redo280_tmp" (using pwrite(..))
6. because we use a RAII Log_file_handle which notices the file was
modified by writes, calls fsync()/fdatasync() before calling close()
on the handle
7. rename "#ib_redo280_tmp" to "#ib_redo280" (using rename() syscall)
8. fsync() or fdatasync() the parent directory which contains the
renamed file (using a handle obtained from open(directory))
9. mark "#ib_redo279" as full (using open(..), pwrite(...) )
(note the lack of fsync. We are ok with persisting it "later",
but it shouldn't be "too early")
10. close("#ib_redo279")
The problem is that in step 8. we fsync wrong directory, so during
recovery the InnoDB would notice that the redo log file with the
largest number, #ib_redo279, already has LOG_HEADER_FLAG_FILE_FULL
indicating that it was full, which implies a new file with even larger
number, #ib_redo280, should have been created (which involves rename)
to accommodate new data, yet this newer file doesn't exist.
InnoDB woud fail with an error:
[ERROR] MY-013893 [InnoDB] Found existing redo log files, but at least
one is missing. It is unknown if recovery could reach physically
consistent state. Please consider restoring from backup or providing
--innodb-force-recovery > 0.
This patch fixes the bug, by calling fsync on the correct directory.
Change-Id: I87f4eb23c60b0d82876789069d1a6716734815ac
This patch revert commit c6953232a38895aae60c37563e12e4476e1d8ade Change-Id: I2c08dbd279958333b86e938cc120b64f5c1c1c5d
Problem: mysqldump_bugs.test failing on release builds Fix: replace --source include/have_debug.inc by mysql_have_debug.inc Change-Id: I151b69dc0da1c5d36714b453ba51349e5057564b
…inux When building with LTO we should use gcc-ar/gcc-ranlib. Look for these versions regardless of LTO flags. Change-Id: I9da5b30bfbd01d4188e8f21721c26fe027e25762 (cherry picked from commit b88d1f86cd768b5555b2da899cbd07e309a156f6)
We have dropped support for some obsolete Linux distros, so we can bump the CMAKE_MINIMUM_REQUIRED VERSION This allows us to use more modern cmake features. Change-Id: I7a37a0f25eff72c00457136dc3b5ee3956b3a139
Post push fix. The mysqltest command replace_result does not work on mysqltest command echo. Change-Id: Iaf81824f723e0ec2cdf9c306f944ed0285b4db46
Some BuildRequires: rules have not been maintained properly. Update version required for cmake and bison. In the patch for 8.0, changed to: -BuildRequires: cmake >= 3.14.6 +BuildRequires: cmake >= 3.11.2 Change-Id: I28041799054bfbc3653e73cfc919aacf4e2b7f7f (cherry picked from commit 33f29fdce4fa3fb71373bbfdf7f27a51514b8009) (cherry picked from commit b0700d0e91abab644515826e156b4070f9de899f)
…tication plugin When the Router is bootstrapped it checks if the metadata user is not using unsupported mysql_native_password plugin. First part of this check is query of the users table for the host and plugin. If this fails the procedure is skipped but the user sees an error messsage as an bootstrap output. Meanwhile this can be no error scenario. For example the user that is used to bootstrap can have no privileges to query users table. This will be the case if the user created by the Shell is used. This patch removes the confusing error message from the bootstrap output. Change-Id: Ic9509a57f5886747f2fe401e566b9f0e2a3c6bd5
Disallow system variable updates during shutdown. Change-Id: I6f259cd8eda8c6ac662573e68bfb17ddc386d1c3
PERFORMANCE_SCHEMA IS OFF Analysis: --------- When performance_schema is set to off, MEB fails with error "Unexpected number of rows from MySQL query" while taking a backup. The tables innodb_redo_log_files & keyring_component_status are not available when performance schema is disabled on the server. When MEB tried to query these tables, it gets empty set as the result and hence it fails. Fix: ---- The fix is to make these tables available even when performance_schema set to OFF or disabled on the server. Change-Id: I0f5fa1b3293733b517f5f19f1dcdebffc651c061
It was reported that the SQL function
`group_replication_set_as_primary()` after successfully set a new
primary remained waiting for the operation completeness.
On the occurrence and implementation analysis, after ruling out
other hypothesis, the only probable cause for the issue is a
missed/not_handled broadcast on a condition wait call.
More precisely on:
```
mysql_mutex_lock(&group_thread_end_lock);
while (action_running && !coordinator_terminating) {
DBUG_PRINT("sleep",
("Waiting for the group action execution process to terminate"));
mysql_cond_wait(&group_thread_end_cond, &group_thread_end_lock);
}
mysql_mutex_unlock(&group_thread_end_lock);
```
which can cause the coordinator of the SQL function
`group_replication_set_as_primary()` to wait uninterruptedly.
To avoid the above issue, the `mysql_cond_wait()` function was
replaced by `mysql_cond_timedwait()` which will periodic check if
the predicate to wait still stands or if the wait can end.
Additionally, a superfluous call to `mysql_cond_broadcast()` was
deleted.
Change-Id: I41fa0f1b3f3ee644e19b916814837ffe5232e265
set after NDBAPI test run
Error insertion improvements
- Have each block log when an error insertion is set
- Indicates previous and new value
- Gives visual indication of when error insert is
set in log
- Info for debugging problems
- Add new ERROR 1 code which can be used to check that
Error inserts are cleared, e.g. at the end of a test
- Improve setting + clearing of error insert and extra
value together.
Change-Id: I18ee88a9a02cec7349a39a85aaec6d03f8df4002
PS-9222 Include reordered fields when calculating mlog buffer size
…cona#5342) https://perconadev.atlassian.net/browse/PS-9165 'keyring_vault.keyring_udf' MTR test case modified so that it could be run on a server built both with and without telemetry component ('-DWITH_PERCONA_TELEMETRY=ON' CMake option).
fields_with_changed_order didn't treat prefix_len key table well. col->get_phy_pos() returns raw data about prefix phy_pos. We need to use field->get_phy_pos() for actual phy_pos here. Change-Id: If13449f9e6e6191cd0f2e7102f62ac28024727f8 (cherry picked from commit 678c193)
Issue: In bug#35183686 fix, we started logging the fields whose column order changed. But while calculating size needed in redo to log index information these columns were missing from calculation. Fix Make sure these colums are also considered while calculating size needed to log index entry. Change-Id: Ic8752c72a8f5beddfc5739688068b9c32b02a700 (cherry picked from commit e6248f5)
- Post push fix for memory leak Change-Id: I034bb20c71dfe3ae467e762b2dd7c7f95fb0679b (cherry picked from commit a3561b3)
https://perconadev.atlassian.net/browse/PS-9222 Our patches have been reverted in favour of Upstream's fixes in 8.0.38. The testcases for the scenario have been preserved.
PS-9222 MySQL server crashes on UPDATE after ALTER TABLE
PS-9284 Release tasks ticket for PS-8.0.37
PS-9284 Release tasks ticket for PS-8.0.37
Release-5.7.44-50
…RCONA_TELEMETRY_DISABLE=1
PS-9320 Please add tarballs for Debian bookworm and Ubuntu Noble for PS
PS-9284 Release tasks ticket for PS-8.0.37
PS-9284 Release tasks ticket for PS 8.0.37
…e-mysql-5.7.44-50
…l-5.7.44-50 Null merge PS 5.7.44-50 at percona@899a9b79e71 into 8.0
https://perconadev.atlassian.net/browse/PS-9233 UUID lib is from dev branch of upcoming version 1.86 The boost::uuid lib is from develop branch, last coomit hash 02c82ce Small fixe for the compatibility fith older boost_1_77_0 (cherry picked from commit 0592d68)
https://perconadev.atlassian.net/browse/PS-9233 This squashed commit contains also fixes done or suggested by Yura Sorokin (cherry picked from commit 63952bf)
…RFC 9562 (packaging) (percona#5360) https://perconadev.atlassian.net/browse/PS-9233 Updated both DEB and RPM packaging scripts to include new 'component_uuid_vx_udf.so' shared library. (cherry picked from commit b0efa52)
oleksandr-kachan
approved these changes
Aug 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.