Skip to content

Commit 9202753

Browse files
authored
New release (#104)
* - Change: drop support for < 8.2 - Change: moved to enums, promoted properties - Added: logger for more socket info - Added: slave_uuid support - Change: config no longer static - Chore: typos in README/code - Chore: replace/remove old urls from code - Chore: changed variables to underscore - Added: support caching_sha2_password - Change: BinLogServerInfo static calls removed also added method getServerInfo to MySQLReplicationFactory
1 parent 95f852a commit 9202753

File tree

88 files changed

+2124
-2831
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

88 files changed

+2124
-2831
lines changed

.github/workflows/tests.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,12 @@ jobs:
1616

1717
strategy:
1818
matrix:
19-
php: [ '7.3', '7.4', '8.0' ]
19+
php: [ '8.2' ]
2020

2121
steps:
2222
- name: Checkout
2323
uses: actions/checkout@v2
24-
24+
2525
- name: Start mysql service
2626
run: |
2727
echo -e "\n[mysqld]\nserver-id=1\nbinlog_format=row\nlog_bin=/var/log/mysql/mysql-bin.log" | sudo tee -a /etc/mysql/my.cnf

.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ composer.phar
55
composer.lock
66
.php_cs.cache
77
/example/profiler.php
8-
.idea/
8+
.idea/
9+
.cache/

.travis.yml

-88
This file was deleted.

.travis/initializedb.sh

-36
This file was deleted.

CHANGELOG.md

+16
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,22 @@
11
# Release Notes
22

3+
## v8.0.0 (2024-01-29)
4+
5+
- Change: drop support for < 8.2
6+
- Change: moved to enums, promoted properties
7+
- Added: logger for more socket info
8+
- Added: slave_uuid support (#99)
9+
- Change: EventInfo->id is now EventInfo->serverId (#83)
10+
- Change: config no longer static (#94)
11+
- Chore: typos in README/code
12+
- Chore: replace/remove old dead doc urls from code
13+
- Chore: changed variables to underscore
14+
- Added: support caching_sha2_password (#102)
15+
- Change: BinLogServerInfo static calls removed also added method getServerInfo to MySQLReplicationFactory
16+
- Change: type of bin log position is now string as it can be bigger then php can hande 2^64-1 (#84)
17+
318
## v7.0.1 (2021-03-09)
19+
420
- Fixed negative number handling (#80)
521

622
## v7.0.0 (2021-01-24)

README.md

+36-16
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
php-mysql-replication
22
=========
3-
[![Build Status](https://travis-ci.org/krowinski/php-mysql-replication.svg?branch=master)](https://travis-ci.org/krowinski/php-mysql-replication)
4-
[![Latest Stable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/stable)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Total Downloads](https://poser.pugx.org/krowinski/php-mysql-replication/downloads)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Latest Unstable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/unstable)](https://packagist.org/packages/krowinski/php-mysql-replication)
5-
[![SensioLabsInsight](https://insight.sensiolabs.com/projects/4a0e49d4-3802-41d3-bb32-0a8194d0fd4d/mini.png)](https://insight.sensiolabs.com/projects/4a0e49d4-3802-41d3-bb32-0a8194d0fd4d) [![License](https://poser.pugx.org/krowinski/php-mysql-replication/license)](https://packagist.org/packages/krowinski/php-mysql-replication)
3+
[![Latest Stable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/stable)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Total Downloads](https://poser.pugx.org/krowinski/php-mysql-replication/downloads)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Latest Unstable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/unstable)](https://packagist.org/packages/krowinski/php-mysql-replication)
64
[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/?branch=master)
75
[![Code Coverage](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/?branch=master)
86

9-
Pure PHP Implementation of MySQL replication protocol. This allow you to receive event like insert, update, delete with their data and raw SQL queries.
7+
Pure PHP Implementation of MySQL replication protocol. This allows you to receive event like insert, update, delete with their data and raw SQL queries.
108

119
Based on a great work of creators:https://github.com/noplay/python-mysql-replication and https://github.com/fengxiangyun/mysql-replication
1210

@@ -29,11 +27,16 @@ composer install -o
2927

3028
Compatibility (based on integration tests)
3129
=========
30+
PHP
3231

32+
- php 8.2
33+
- php 8.3
34+
35+
MYSQL
3336
- mysql 5.5
3437
- mysql 5.6
3538
- mysql 5.7
36-
- mysql 8.0 (ONLY with mysql_native_password)
39+
- mysql 8.0 (mysql_native_password and caching_sha2_password supported)
3740
- mariadb 5.5
3841
- mariadb 10.0
3942
- mariadb 10.1
@@ -83,7 +86,7 @@ Available options:
8386

8487
'mariaDbGtid' - MariaDB GTID marker(s) to start from (format 1-1-3,0-1-88)
8588

86-
'slaveId' - script slave id for identification (SHOW SLAVE HOSTS)
89+
'slaveId' - script slave id for identification (default: 666) (SHOW SLAVE HOSTS)
8790

8891
'binLogFileName' - bin log file name to start from
8992

@@ -103,6 +106,8 @@ Available options:
103106

104107
'heartbeatPeriod' - sets the interval in seconds between replication heartbeats. Whenever the master's binary log is updated with an event, the waiting period for the next heartbeat is reset. interval is a decimal value having the range 0 to 4294967 seconds and a resolution in milliseconds; the smallest nonzero value is 0.001. Heartbeats are sent by the master only if there are no unsent events in the binary log file for a period longer than interval.
105108

109+
'saveUuid' - sets slave uuid for identification (default: 0015d2b6-8a06-4e5e-8c07-206ef3fbd274)
110+
106111
Similar projects
107112
=========
108113
Ruby: https://github.com/y310/kodama
@@ -394,27 +399,42 @@ FAQ
394399
=========
395400

396401
1. ### Why and when need php-mysql-replication ?
397-
Well first of all mysql don't give you async calls. You usually need to program this in your application (by event dispaching and adding to some queue system and if your db have many point of entry like web, backend other microservices its not always cheap to add processing to all of them. But using mysql replication protocol you can lisen on write events and process then asynchronously (best combo it's to add item to some queue system like rabbitmq, redis or kafka). Also in invalidate cache, search engine replication, real time analytics and audits.
398402

399-
2. ### It's awsome ! but what is the catch ?
400-
Well first of all you need to know that a lot of events may come through, like if you update 1 000 000 records in table "bar" and you need this one insert from your table "foo" Then all must be processed by script and you need to wait for your data. This is normal and this how it's work. You can speed up using [config options](https://github.com/krowinski/php-mysql-replication#configuration).
401-
Also if script crashes you need to save from time to time position form binlog (or gtid) to start from this position when you run this script again to avoid duplicates.
403+
Well first of all MYSQL don't give you async calls. You usually need to program this in your application (by event dispatching and adding to some queue system
404+
and if your db have many point of entry like web, backend other microservices its not always cheap to add processing to all of them. But using mysql replication
405+
protocol you can listen on write events and process then asynchronously (the best combo it's to add item to some queue system like rabbitmq, redis or kafka).
406+
Also in invalidate cache, search engine replication, real time analytics and audits.
407+
408+
2. ### It's awesome ! but what is the catch ?
409+
410+
Well first of all you need to know that a lot of events may come through, like if you update 1 000 000 records in table "bar" and you need this one insert from
411+
your table "foo" Then all must be processed by script, and you need to wait for your data. This is normal and this how it's work. You can speed up
412+
using [config options](https://github.com/krowinski/php-mysql-replication#configuration).
413+
Also, if script crashes you need to save from time to time position form binlog (or gtid) to start from this position when you run this script again to avoid
414+
duplicates.
402415

403416
3. ### I need to process 1 000 000 records and its taking forever!!
404417
Like I mention in 1 point use queue system like rabbitmq, redis or kafka, they will give you ability to process data in multiple scripts.
405418

406419
4. ### I have a problem ? you script is missing something ! I have found a bug !
407420
Create an [issue](https://github.com/krowinski/php-mysql-replication/issues) I will try to work on it in my free time :)
408421

409-
5. ### How much its give overhead to mysql server ?
410-
It work like any other mysql in slave mode and its giving same overhead.
422+
5. ### How much its give overhead to MYSQL server ?
423+
424+
It work like any other MYSQL in slave mode and its giving same overhead.
411425

412426
6. ### Socket timeouts error
413-
To fix this best is to increase db configurations "net_read_timeout" and "net_write_timeout" to 3600. (tx Bijimon)
427+
428+
To fix this best is to increase db configurations ```net_read_timeout``` and ```net_write_timeout``` to 3600. (tx Bijimon)
414429

415430
7. ### Partial updates fix
416-
Set in my.conf ```binlog_row_image=full``` to fix reciving only partial updates.
417-
431+
432+
Set in my.conf ```binlog_row_image=full``` to fix receiving only partial updates.
433+
418434
8. ### No replication events when connected to replica server
419-
Set in my.conf ```log_slave_updates=on``` to fix this (#71)(#66)
435+
Set in my.conf ```log_slave_updates=on``` to fix this (#71)(#66)
420436

437+
9. ### "Big" updates / inserts
438+
Default MYSQL setting generates one big blob of stream this require more RAM/CPU you can change this for smaller stream using
439+
variable ```binlog_row_event_max_size``` [https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_row_event_max_size] to
440+
split into smaller chunks

composer.json

+21-7
Original file line numberDiff line numberDiff line change
@@ -12,18 +12,23 @@
1212
],
1313
"type": "library",
1414
"require": {
15-
"php": ">=7.3",
15+
"php": ">=8.2",
1616
"ext-bcmath": "*",
1717
"ext-json": "*",
1818
"ext-sockets": "*",
19-
"doctrine/collections": "^1.3",
20-
"doctrine/dbal": "^3.0",
21-
"psr/simple-cache": "^1.0",
22-
"symfony/dependency-injection": "^3.1|^4.0|^5.0",
23-
"symfony/event-dispatcher": "^3.1|^4.0|^5.0"
19+
"doctrine/collections": "^2.1",
20+
"doctrine/dbal": "^3.8",
21+
"psr/log": "^3.0",
22+
"psr/simple-cache": "^3.0",
23+
"symfony/dependency-injection": "^7.0",
24+
"symfony/event-dispatcher": "^7.0"
2425
},
2526
"require-dev": {
26-
"phpunit/phpunit": "^9.0"
27+
"kubawerlos/php-cs-fixer-custom-fixers": "^3.19",
28+
"monolog/monolog": "^3.5",
29+
"phpstan/phpstan": "^1.10",
30+
"phpunit/phpunit": "^10.5",
31+
"symplify/easy-coding-standard": "^12.1"
2732
},
2833
"license": "MIT",
2934
"authors": [
@@ -45,5 +50,14 @@
4550
"minimum-stability": "stable",
4651
"config": {
4752
"sort-packages": true
53+
},
54+
"scripts": {
55+
"cs:check": "ecs check",
56+
"cs:fix": "ecs check --fix",
57+
"phpstan:analyse": "phpstan analyse -cphpstan.neon",
58+
"sa": [
59+
"@cs:check",
60+
"@phpstan:analyse"
61+
]
4862
}
4963
}

docker-compose.yml

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
services:
2+
replication-test-mysql-percona:
3+
container_name: replication-test-mysql-percona
4+
hostname: replication-test-mysql-percona
5+
image: mysql:8.0
6+
platform: linux/amd64
7+
command: [
8+
'--character-set-server=utf8mb4',
9+
'--collation-server=utf8mb4_unicode_ci',
10+
# '--default-authentication-plugin=caching_sha2_password',
11+
#'--default-authentication-plugin=mysql_native_password',
12+
'--log_bin=binlog',
13+
'--max_binlog_size=8M',
14+
'--binlog_format=row',
15+
'--server-id=1'
16+
]
17+
environment:
18+
- MYSQL_ROOT_PASSWORD=root
19+
- MYSQL_DATABASE=mysqlreplication_test
20+
ports:
21+
- "3306:3306/tcp"
22+
restart: unless-stopped

ecs.php

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
use PhpCsFixer\Fixer\ArrayNotation\ReturnToYieldFromFixer;
6+
use PhpCsFixer\Fixer\ArrayNotation\YieldFromArrayToYieldsFixer;
7+
use PhpCsFixer\Fixer\Casing\NativeTypeDeclarationCasingFixer;
8+
use PhpCsFixer\Fixer\FunctionNotation\PhpdocToParamTypeFixer;
9+
use PhpCsFixer\Fixer\FunctionNotation\PhpdocToPropertyTypeFixer;
10+
use PhpCsFixer\Fixer\FunctionNotation\PhpdocToReturnTypeFixer;
11+
use PhpCsFixer\Fixer\Phpdoc\PhpdocAddMissingParamAnnotationFixer;
12+
use PhpCsFixer\Fixer\Whitespace\TypeDeclarationSpacesFixer;
13+
use PhpCsFixerCustomFixers\Fixer\NoLeadingSlashInGlobalNamespaceFixer;
14+
use PhpCsFixerCustomFixers\Fixer\PhpdocNoSuperfluousParamFixer;
15+
use PhpCsFixerCustomFixers\Fixer\PromotedConstructorPropertyFixer;
16+
use Symplify\EasyCodingStandard\Config\ECSConfig;
17+
use Symplify\EasyCodingStandard\ValueObject\Set\SetList;
18+
19+
return static function (ECSConfig $ecsConfig): void {
20+
$ecsConfig->parallel();
21+
$ecsConfig->sets([
22+
SetList::PSR_12,
23+
SetList::CLEAN_CODE,
24+
SetList::STRICT,
25+
SetList::ARRAY,
26+
SetList::PHPUNIT,
27+
SetList::DOCTRINE_ANNOTATIONS,
28+
SetList::COMMENTS,
29+
SetList::SYMPLIFY,
30+
SetList::CONTROL_STRUCTURES,
31+
]);
32+
33+
$ecsConfig->rules([
34+
NativeTypeDeclarationCasingFixer::class,
35+
ReturnToYieldFromFixer::class,
36+
TypeDeclarationSpacesFixer::class,
37+
YieldFromArrayToYieldsFixer::class,
38+
PhpdocToPropertyTypeFixer::class,
39+
PhpdocToParamTypeFixer::class,
40+
PhpdocToReturnTypeFixer::class,
41+
PromotedConstructorPropertyFixer::class,
42+
NoLeadingSlashInGlobalNamespaceFixer::class,
43+
PhpdocNoSuperfluousParamFixer::class,
44+
PhpdocAddMissingParamAnnotationFixer::class,
45+
]);
46+
47+
$ecsConfig->fileExtensions(['php']);
48+
$ecsConfig->cacheDirectory('.cache/ecs');
49+
$ecsConfig->paths([__DIR__ . '/src', __DIR__ . '/tests', __DIR__ . '/example',]);
50+
};

0 commit comments

Comments
 (0)