使用SQL語句操作PolarDB-X的binlog
SQL介紹
PolarDB-X提供了兼容MySQL的SQL語法,可以對binlog文件進(jìn)行查看和操作。
多流binlog的SQL兼容
如上所列的SHOW BINARY LOGS、SHOW BINLOG EVENTS、SHOW MASTER STATUS是最常用的3個SQL命令,適配MySQL binlog的很多生態(tài)工具在程序代碼中也多有使用。
針對多流binlog,PolarDB-X對這3個SQL命令進(jìn)行了語法擴展,可以通過指定with選項,針對特定的流或流組進(jìn)行操作。但眾多的生態(tài)工具并不支持該擴展語法,推進(jìn)生態(tài)工具適配PolarDB-X的擴展語法也需要較長的實施周期(Alibaba canal已經(jīng)實現(xiàn)了適配),對此,PolarDB-X提供了一種將賬號和binlog流進(jìn)行綁定的機制,來實現(xiàn)0改造即可使用binlog多流服務(wù)。
使用限制
大于等于5.4.19的版本才支持該綁定機制。
如果不想開啟綁定機制,在PolarDB-X命令行執(zhí)行
set global ENABLE_EXTRACT_STREAM_NAME_FROM_USER = false
。
使用方法
通過show binary stream查看所有的流信息。
show binary streams; +--------+-----------------+-------------------------------+----------+ | GROUP | STREAM | FILE | POSITION | +--------+-----------------+-------------------------------+----------+ | group1 | group1_stream_0 | group1_stream_0_binlog.000438 | 6690 | | group1 | group1_stream_1 | group1_stream_1_binlog.000440 | 6690 | | group1 | group1_stream_2 | group1_stream_2_binlog.000452 | 6690 | | group1 | group1_stream_3 | group1_stream_3_binlog.000449 | 6690 | +--------+-----------------+-------------------------------+----------+
為每條流創(chuàng)建一個綁定賬號并授予合適的權(quán)限,賬號的命名規(guī)則為: 流名稱 + _cdc_user后綴,如下所示
CREATE USER IF NOT EXISTS 'group1_stream_0_cdc_user'@'%' identified by '123456'; CREATE USER IF NOT EXISTS 'group1_stream_1_cdc_user'@'%' identified by '123456'; CREATE USER IF NOT EXISTS 'group1_stream_2_cdc_user'@'%' identified by '123456'; CREATE USER IF NOT EXISTS 'group1_stream_3_cdc_user'@'%' identified by '123456'; grant ALL PRIVILEGES on *.* to 'group1_stream_0_cdc_user'@'%'; grant ALL PRIVILEGES on *.* to 'group1_stream_1_cdc_user'@'%'; grant ALL PRIVILEGES on *.* to 'group1_stream_2_cdc_user'@'%'; grant ALL PRIVILEGES on *.* to 'group1_stream_3_cdc_user'@'%';
使用創(chuàng)建的賬號登錄PolarDB-X,并執(zhí)行SHOW BINARY LOGS、SHOW BINLOG EVENTS、SHOW MASTER STATUS,對返回內(nèi)容進(jìn)行驗證,返回內(nèi)容如果對應(yīng)的是當(dāng)前登錄賬號中包含的binlog流的信息,則符合預(yù)期。
示例
SHOW BINARY LOGS
使用普通賬號執(zhí)行show binary logs
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show binary logs; +---------------+-----------+ | LOG_NAME | FILE_SIZE | +---------------+-----------+ | binlog.000001 | 260 | | binlog.000002 | 6031 | | binlog.000003 | 3046 | | binlog.000004 | 6827 | | binlog.000005 | 3046 | | binlog.000006 | 612906 | | binlog.000007 | 3440227 | | binlog.000008 | 10486619 | | binlog.000009 | 6825583 | | binlog.000010 | 3245 | +---------------+-----------+
使用普通賬號加with選項執(zhí)行show binary logs
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show binary logs with 'group1_stream_0'; +-------------------------------+-----------+ | LOG_NAME | FILE_SIZE | +-------------------------------+-----------+ | group1_stream_0_binlog.000001 | 276 | | group1_stream_0_binlog.000002 | 8634 | | group1_stream_0_binlog.000003 | 9629 | | group1_stream_0_binlog.000004 | 615708 | | group1_stream_0_binlog.000005 | 8775293 | | group1_stream_0_binlog.000006 | 10027 | | group1_stream_0_binlog.000007 | 9430 | | group1_stream_0_binlog.000008 | 13437 | | group1_stream_0_binlog.000009 | 10049 | | group1_stream_0_binlog.000010 | 9629 | +-------------------------------+-----------+
使用綁定賬號執(zhí)行show binary logs
select user(); +------------------------------------+ | USER() | +------------------------------------+ | group1_stream_0_cdc_user@127.0.0.1 | +------------------------------------+ 1 row in set (0.00 sec) show binary logs; +-------------------------------+-----------+ | LOG_NAME | FILE_SIZE | +-------------------------------+-----------+ | group1_stream_0_binlog.000001 | 276 | | group1_stream_0_binlog.000002 | 8634 | | group1_stream_0_binlog.000003 | 9629 | | group1_stream_0_binlog.000004 | 615708 | | group1_stream_0_binlog.000005 | 8775293 | | group1_stream_0_binlog.000006 | 10027 | | group1_stream_0_binlog.000007 | 9430 | | group1_stream_0_binlog.000008 | 13437 | | group1_stream_0_binlog.000009 | 10049 | | group1_stream_0_binlog.000010 | 9629 | +-------------------------------+-----------+
SHOW MASTER STATUS
使用普通賬號執(zhí)行show master status
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show master status; +---------------+----------+--------------+------------------+-------------------+ | FILE | POSITION | BINLOG_DO_DB | BINLOG_IGNORE_DB | EXECUTED_GTID_SET | +---------------+----------+--------------+------------------+-------------------+ | binlog.001219 | 4899 | | | | +---------------+----------+--------------+------------------+-------------------+
使用普通賬號加with選項執(zhí)行show master status
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show master status with 'group1_stream_0'; +-------------------------------+----------+--------------+------------------+-------------------+ | FILE | POSITION | BINLOG_DO_DB | BINLOG_IGNORE_DB | EXECUTED_GTID_SET | +-------------------------------+----------+--------------+------------------+-------------------+ | group1_stream_0_binlog.000442 | 10840 | | | | +-------------------------------+----------+--------------+------------------+-------------------+
使用綁定賬號執(zhí)行show master status
select user(); +------------------------------------+ | USER() | +------------------------------------+ | group1_stream_0_cdc_user@127.0.0.1 | +------------------------------------+ 1 row in set (0.00 sec) show master status; +-------------------------------+----------+--------------+------------------+-------------------+ | FILE | POSITION | BINLOG_DO_DB | BINLOG_IGNORE_DB | EXECUTED_GTID_SET | +-------------------------------+----------+--------------+------------------+-------------------+ | group1_stream_0_binlog.000443 | 1118 | | | | +-------------------------------+----------+--------------+------------------+-------------------+
SHOW BINLOG EVENTS
使用普通賬號執(zhí)行show binlog events
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show binlog events limit 5; +---------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | LOG_NAME | POS | EVENT_TYPE | SERVER_ID | END_LOG_POS | INFO | +---------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | binlog.000001 | 4 | Format_desc | 3189545694 | 123 | Server ver: 5.6.29-TDDL-5.4.19-SNAPSHOT, Binlog ver: 4 | | binlog.000001 | 123 | Rows_query | 3189545694 | 216 | CTS::718558471351435270417166499290336542720000000000000000::FlushLog | | binlog.000001 | 216 | Rotate | 3189545694 | 260 | binlog.000002;pos=4 | +---------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+
使用普通賬號加with選項執(zhí)行show binlog events
select user(); +-------------------------+ | USER() | +-------------------------+ | polardbx_root@127.0.0.1 | +-------------------------+ 1 row in set (0.00 sec) show binlog events with 'group1_stream_0' limit 5; +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | LOG_NAME | POS | EVENT_TYPE | SERVER_ID | END_LOG_POS | INFO | +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | group1_stream_0_binlog.000001 | 4 | Format_desc | 3189545694 | 123 | Server ver: 5.6.29-TDDL-5.4.19-SNAPSHOT, Binlog ver: 4 | | group1_stream_0_binlog.000001 | 123 | Rows_query | 3189545694 | 216 | CTS::718558434551031404817166495609667010560000000000000000::FlushLog | | group1_stream_0_binlog.000001 | 216 | Rotate | 3189545694 | 276 | group1_stream_0_binlog.000002;pos=4 | +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+
使用綁定賬號執(zhí)行show binlog events
select user(); +------------------------------------+ | USER() | +------------------------------------+ | group1_stream_0_cdc_user@127.0.0.1 | +------------------------------------+ 1 row in set (0.00 sec) show binlog events limit 5; +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | LOG_NAME | POS | EVENT_TYPE | SERVER_ID | END_LOG_POS | INFO | +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+ | group1_stream_0_binlog.000001 | 4 | Format_desc | 3189545694 | 123 | Server ver: 5.6.29-TDDL-5.4.19-SNAPSHOT, Binlog ver: 4 | | group1_stream_0_binlog.000001 | 123 | Rows_query | 3189545694 | 216 | CTS::718558434551031404817166495609667010560000000000000000::FlushLog | | group1_stream_0_binlog.000001 | 216 | Rotate | 3189545694 | 276 | group1_stream_0_binlog.000002;pos=4 | +-------------------------------+------+-------------+------------+-------------+-----------------------------------------------------------------------+