在線分區(qū)維護(hù)
本文檔主要介紹了PolarDB的在線分區(qū)維護(hù)功能。
在MySQL社區(qū)版中,對(duì)分區(qū)表數(shù)據(jù)的訪問(wèn)操作(DML)與分區(qū)維護(hù)操作(DDL)是相互阻塞的,這使得分區(qū)維護(hù)操作只能在業(yè)務(wù)低峰時(shí)段進(jìn)行。而對(duì)分區(qū)表進(jìn)行創(chuàng)建和刪除分區(qū)操作是比較頻繁的,這極大限制了分區(qū)表的使用。在線分區(qū)維護(hù)功能增強(qiáng)了分區(qū)的DML和特定DDL(增加、刪除分區(qū))的并行能力,更好的讓用戶對(duì)分區(qū)表進(jìn)行Roll-In和Roll-Out。如下圖所示:
前提條件
集群版本需為PolarDB MySQL版8.0版本且Revision version為8.0.2.2.0及以上。您可以通過(guò)查詢版本號(hào)確認(rèn)集群版本。
已經(jīng)開啟partition_level_mdl_enabled參數(shù)。具體操作請(qǐng)參見設(shè)置集群參數(shù)和節(jié)點(diǎn)參數(shù)。
參數(shù)名稱
級(jí)別
參數(shù)說(shuō)明
partition_level_mdl_enabled
Global
PARTITION Level MDL功能開關(guān),取值范圍如下:
ON:開啟PARTITION Level MDL功能
OFF:關(guān)閉PARTITION Level MDL功能
說(shuō)明修改該參數(shù)需要重啟集群。
transaction_isolation參數(shù)的全局隔離級(jí)別需要設(shè)置為READ-COMMITTED。具體操作請(qǐng)參見設(shè)置集群參數(shù)和節(jié)點(diǎn)參數(shù)。
使用限制
目前支持DROP/EXCHANGE/REBUILD/REORGANIZE PARTITION操作、RANGE和LIST分區(qū)方式的ADD PARTITION操作的在線分區(qū)維護(hù)功能,其他DDL操作將在后續(xù)版本支持。
由于隔離級(jí)別可以設(shè)置為session級(jí)別,如果transaction-isolation設(shè)置為REPEATABLE-READ或者更強(qiáng)的隔離級(jí)別,在并發(fā)執(zhí)行DDL過(guò)程中,可能會(huì)報(bào)錯(cuò)
ERROR HY000: Table definition has changed, please retry transaction
,這屬于正常現(xiàn)象。原因是事務(wù)訪問(wèn)到了DDL所創(chuàng)建出來(lái)的新分區(qū),可通過(guò)重新執(zhí)行事務(wù)解決。
使用說(shuō)明及示例
在線分區(qū)維護(hù)功能,使得分區(qū)表的數(shù)據(jù)訪問(wèn)和分區(qū)維護(hù)相互不影響,用戶可以更自由的進(jìn)行分區(qū)維護(hù),而不影響分區(qū)表業(yè)務(wù)流量,使用示例及效果如下:
#在客戶端1上輸出tr表結(jié)構(gòu)
SHOW CREATE TABLE tr\G
*************************** 1. row ***************************
Table: tr
Create Table: CREATE TABLE `tr` (
`id` int(11) DEFAULT NULL,
`name` varchar(50) DEFAULT NULL,
`purchased` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
/*!50100 PARTITION BY RANGE (year(`purchased`))
(PARTITION p0 VALUES LESS THAN (1990) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (1995) ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN (2000) ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN (2005) ENGINE = InnoDB,
PARTITION p4 VALUES LESS THAN (2010) ENGINE = InnoDB,
PARTITION p5 VALUES LESS THAN (2015) ENGINE = InnoDB) */
1 row in set (0.00 sec)
#在客戶端1上開啟事務(wù)
BEGIN;
Query OK, 0 rows affected (0.01 sec)
SELECT * FROM tr WHERE purchased >= '2010-01-01';
+------+----------------+------------+
| id | name | purchased |
+------+----------------+------------+
| 5 | exercise bike | 2014-05-09 |
| 7 | espresso maker | 2011-11-22 |
+------+----------------+------------+
2 rows in set (0.01 sec)
#在客戶端2上創(chuàng)建新分區(qū)
ALTER TABLE tr ADD PARTITION (PARTITION p6 VALUES LESS THAN (2020));
INSERT INTO tr VALUES (11, 'hope', '2017-11-04'), (12, 'carmen', '2018-06-08');
#在同一個(gè)事務(wù)內(nèi),通過(guò)客戶端1可以訪問(wèn)到新增分區(qū)的數(shù)據(jù)
SELECT * FROM tr WHERE purchased >= '2010-01-01';
+------+----------------+------------+
| id | name | purchased |
+------+----------------+------------+
| 5 | exercise bike | 2014-05-09 |
| 7 | espresso maker | 2011-11-22 |
| 11 | hope | 2017-11-04 |
| 12 | carmen | 2018-06-08 |
+------+----------------+------------+
4 rows in set (0.00 sec)
#在客戶端2上刪除舊分區(qū)
ALTER TABLE tr DROP PARTITION p0;
#在客戶端1上輸出表定義,可以看到新分區(qū)p6存在,舊分區(qū)p0已不存在
SHOW CREATE TABLE tr\G
*************************** 1. row ***************************
Table: tr
Create Table: CREATE TABLE `tr` (
`id` int(11) DEFAULT NULL,
`name` varchar(50) DEFAULT NULL,
`purchased` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
/*!50100 PARTITION BY RANGE (year(`purchased`))
(PARTITION p1 VALUES LESS THAN (1995) ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN (2000) ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN (2005) ENGINE = InnoDB,
PARTITION p4 VALUES LESS THAN (2010) ENGINE = InnoDB,
PARTITION p5 VALUES LESS THAN (2015) ENGINE = InnoDB,
PARTITION p6 VALUES LESS THAN (2020) ENGINE = InnoDB) */
1 row in set (0.00 sec)
#在客戶端1上提交事務(wù)
COMMIT;
性能表現(xiàn)
下面針對(duì)DML和DDL交互下的兩種最主要場(chǎng)景長(zhǎng)事務(wù)阻塞DDL和耗時(shí)DDL,在分別開啟和關(guān)閉在線分區(qū)維護(hù)功能的情況下,進(jìn)行性能對(duì)比驗(yàn)證。
場(chǎng)景一:長(zhǎng)事務(wù)阻塞DDL
此場(chǎng)景下對(duì)分區(qū)表做分區(qū)維護(hù)DDL操作,由于存在未提交的長(zhǎng)事務(wù),從而阻塞DDL操作的進(jìn)行。被阻塞的DDL操作會(huì)進(jìn)一步阻塞所有新產(chǎn)生的DML操作,導(dǎo)致了數(shù)據(jù)庫(kù)流量跌零。
從上圖可以看到,在功能關(guān)閉時(shí),當(dāng)DDL操作發(fā)生后,sysbench在該表的流量迅速跌零,數(shù)據(jù)庫(kù)呈現(xiàn)完全不可用的狀態(tài)。用戶只能通過(guò)取消DDL操作或者提交所有未提交的長(zhǎng)事務(wù)后,數(shù)據(jù)庫(kù)才能恢復(fù)正常;在功能開啟后,可以看到:
在正常情況下,sysbench的吞吐量與在線分區(qū)維護(hù)功能關(guān)閉時(shí)完全一致,說(shuō)明在線分區(qū)維護(hù)功能的引入并沒有帶來(lái)性能方面的損耗;
未提交的長(zhǎng)事務(wù)并不會(huì)阻塞分區(qū)維護(hù)操作的執(zhí)行,同時(shí)數(shù)據(jù)庫(kù)上的DML流量很穩(wěn)定,幾乎沒有受到影響。
場(chǎng)景二:耗時(shí)DDL
該場(chǎng)景下,分區(qū)維護(hù)DDL操作雖然沒有受到其他SQL語(yǔ)句的阻塞,但是在本身執(zhí)行分區(qū)維護(hù)DDL操作耗時(shí)的情況下,對(duì)DML吞吐量有影響。
從上圖可以看到,在功能關(guān)閉時(shí),長(zhǎng)耗時(shí)DDL操作在執(zhí)行期間導(dǎo)致了DML吞吐量的劇烈抖動(dòng);在功能開啟后,長(zhǎng)耗時(shí)DDL操作在執(zhí)行期間對(duì) DML的吞吐量幾乎沒有影響。
查看MDL鎖狀態(tài)
在線分區(qū)維護(hù)功能是通過(guò)引入分區(qū)級(jí)別的MDL鎖,來(lái)降低在DML和DDL過(guò)程中獲取的鎖粒度,從而增加并發(fā)性來(lái)實(shí)現(xiàn)的。在分區(qū)維護(hù)過(guò)程中,通過(guò)performance_schema.metadata_locks
表可以查看分區(qū)級(jí)別的MDL鎖獲取情況。示例如下:
#在客戶端1上輸出tr表結(jié)構(gòu)
SHOW CREATE TABLE tr\G
*************************** 1. row ***************************
Table: tr
Create Table: CREATE TABLE `tr` (
`id` int(11) DEFAULT NULL,
`name` varchar(50) DEFAULT NULL,
`purchased` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
/*!50100 PARTITION BY RANGE (year(`purchased`))
(PARTITION p0 VALUES LESS THAN (1990) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (1995) ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN (2000) ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN (2005) ENGINE = InnoDB,
PARTITION p4 VALUES LESS THAN (2010) ENGINE = InnoDB,
PARTITION p5 VALUES LESS THAN (2015) ENGINE = InnoDB) */
1 row in set (0.00 sec)
#在客戶端1上開啟事務(wù)
BEGIN;
Query OK, 0 rows affected (0.01 sec)
SELECT * FROM tr WHERE purchased >= '2010-01-01';
+------+----------------+------------+
| id | name | purchased |
+------+----------------+------------+
| 5 | exercise bike | 2014-05-09 |
| 7 | espresso maker | 2011-11-22 |
+------+----------------+------------+
2 rows in set (0.01 sec)
#在客戶端1上查看MDL鎖的狀態(tài)
SELECT * FROM performance_schema.metadata_locks;
+-------------+--------------------+----------------+-------------+-----------------------+---------------------+---------------+-------------+-------------------+-----------------+----------------+
| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME | COLUMN_NAME | OBJECT_INSTANCE_BEGIN | LOCK_TYPE | LOCK_DURATION | LOCK_STATUS | SOURCE | OWNER_THREAD_ID | OWNER_EVENT_ID |
+-------------+--------------------+----------------+-------------+-----------------------+---------------------+---------------+-------------+-------------------+-----------------+----------------+
| TABLE | test | tr | NULL | 140734887898944 | SHARED_READ | TRANSACTION | GRANTED | sql_parse.cc:6759 | 67 | 17 |
| PARTITION | test | tr | p5 | 140734887896704 | SHARED_READ | TRANSACTION | GRANTED | sql_parse.cc:6502 | 67 | 17 |
| TABLE | performance_schema | metadata_locks | NULL | 140734879511488 | SHARED_READ | TRANSACTION | GRANTED | sql_parse.cc:6759 | 68 | 4 |
| SCHEMA | performance_schema | NULL | NULL | 140734879511648 | INTENTION_EXCLUSIVE | TRANSACTION | GRANTED | dd_schema.cc:108 | 68 | 4 |
+-------------+--------------------+----------------+-------------+-----------------------+---------------------+---------------+-------------+-------------------+-----------------+----------------+
4 rows in set (0.02 sec)
#在客戶端1上可以看到,獲得tr表級(jí)的SHARED_READ鎖和剪枝后實(shí)際需要訪問(wèn)p5分區(qū)級(jí)別的SHARED_READ鎖。接下來(lái),通過(guò)客戶端2嘗試將分區(qū)p5刪除
ALTER TABLE tr DROP PARTITION p5;
#由于分區(qū)p5正在被客戶端1問(wèn),所以上述操作會(huì)進(jìn)入等待狀態(tài),通過(guò)下述命令可以看到該線程正在等待分區(qū)級(jí)別的MDL鎖。
SHOW PROCESSLIST;
+----+-----------------+-----------+------+---------+------+-------------------------------------+----------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+-----------+------+---------+------+-------------------------------------+----------------------------------+
| 4 | event_scheduler | localhost | NULL | Daemon | 1550 | Waiting on empty queue | NULL |
| 8 | root | localhost | test | Sleep | 426 | | NULL |
| 9 | root | localhost | NULL | Query | 0 | starting | SHOW PROCESSLIST |
| 10 | root | localhost | test | Query | 10 | Waiting for partition metadata lock | ALTER TABLE tr DROP PARTITION p5 |
+----+-----------------+-----------+------+---------+------+-------------------------------------+----------------------------------+
4 rows in set (0.00 sec)
#通過(guò)客戶端1提交事務(wù)后,客戶端2能夠順利完成分區(qū)的維護(hù)。
查看在線分區(qū)維護(hù)次數(shù)
通過(guò)STATUS變量Online_altered_partition
參數(shù)可以監(jiān)控進(jìn)行在線分區(qū)維護(hù)的次數(shù),示例如下:
SHOW STATUS LIKE 'Online_altered_partition';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| Online_altered_partition | 2565 |
+--------------------------+-------+
1 row in set (0.00 sec)