創(chuàng)建DLM策略

以下示例將創(chuàng)建sales分區(qū)表，該表以order_time列作為分區(qū)鍵，按時間間隔劃分分區(qū)。同時，該表存在INTERVAL和DLM兩種策略：

• INTERVAL策略：當插入的數(shù)據(jù)超過分區(qū)范圍時，將自動創(chuàng)建新的分區(qū)，時間間隔為1年。

• DLM策略：定義當前表僅有3個分區(qū)，當分區(qū)數(shù)量大于3，執(zhí)行DLM策略時：

  • 如果不存在OSS外表sales_history，則創(chuàng)建新的OSS外表sales_history，并將冷數(shù)據(jù)轉(zhuǎn)存到sales_history外表中。

  • 如果存在sales_history外表，且sales_history表在內(nèi)置的OSS 空間上，則直接將冷數(shù)據(jù)轉(zhuǎn)存到sales_history外表中。

1. 創(chuàng)建帶有DLM策略的sales表。

   CREATE TABLE `sales` (
     `id` int DEFAULT NULL,
     `name` varchar(20) DEFAULT NULL,
     `order_time` datetime NOT NULL,
      PRIMARY KEY (`order_time`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
   PARTITION BY RANGE  COLUMNS(order_time) INTERVAL(YEAR, 1)
   (PARTITION p20200101000000 VALUES LESS THAN ('2020-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20210101000000 VALUES LESS THAN ('2021-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20220101000000 VALUES LESS THAN ('2022-01-01 00:00:00') ENGINE = InnoDB)
   DLM ADD POLICY test_policy TIER TO TABLE ENGINE=CSV STORAGE=OSS READ ONLY
   STORAGE TABLE_NAME = 'sales_history' EXTRA_INFO '{"oss_file_filter":"id,name:bloom"}' ON (PARTITIONS OVER 3);

   該表的DLM策略名稱為test_policy，當分區(qū)數(shù)量大于3時，會將當前表的冷數(shù)據(jù)轉(zhuǎn)存為CSV格式，并存儲在OSS上。轉(zhuǎn)存后的表名稱為sales_history，數(shù)據(jù)只讀。同時，如果歸檔表OSS不存在，會自動創(chuàng)建OSS表，并在id和name列上創(chuàng)建OSS_FILE_FILTER。

2. 當前表的DLM策略會存儲在系統(tǒng)表mysql.dlm_policies中，您可以通過該表查看DLM策略的詳細信息。表mysql.dlm_policies詳情請參見表結(jié)構說明。查看mysql.dlm_policies表結(jié)構信息。

   mysql> SELECT * FROM mysql.dlm_policies\G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************
                      Id: 3
            Table_schema: gg
              Table_name: sales
             Policy_name: test_policy
             Policy_type: TABLE
            Archive_type: PARTITION COUNT
            Storage_mode: READ ONLY
          Storage_engine: CSV
           Storage_media: OSS
     Storage_schema_name: gg
      Storage_table_name: sales_history
         Data_compressed: OFF
    Compressed_algorithm: NULL
                 Enabled: ENABLED
         Priority_number: 10300
   Tier_partition_number: 3
          Tier_condition: NULL
              Extra_info: {"oss_file_filter": "id,name:bloom,order_time"}
                 Comment: NULL
   1 row in set (0.03 sec)      

   目前sales表的分區(qū)數(shù)量為3，不會進行數(shù)據(jù)歸檔。

3. 向sales分區(qū)表中插入3000的測試數(shù)據(jù)，保證數(shù)據(jù)超過當前表的分區(qū)定義，觸發(fā)INTERVAL自動創(chuàng)建新的分區(qū)。

   DROP PROCEDURE IF EXISTS proc_batch_insert;
   delimiter $$
   CREATE PROCEDURE proc_batch_insert(IN begin INT, IN end INT, IN name VARCHAR(20))
   BEGIN
   SET @insert_stmt = concat('INSERT INTO ', name, ' VALUES(? , ?, ?);');
   PREPARE stmt from @insert_stmt;
   WHILE begin <= end DO
   SET @ID1 = begin;
   SET @NAME = CONCAT(begin+begin*281313, '@stiven');
   SET @TIME = from_days(begin + 737600);
   EXECUTE stmt using @ID1, @NAME, @TIME;
   SET begin = begin + 1;
   END WHILE;
   END;
   $$
   delimiter ;
   CALL proc_batch_insert(1, 3000, 'sales');

4. 此時，觸發(fā)了INTERVAL自動創(chuàng)建新的分區(qū)，導致sales表的分區(qū)增加，表結(jié)構變?yōu)椋?/p>

   mysql> SHOW CREATE TABLE sales\G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************       
         Table: sales
   Create Table: CREATE TABLE `sales` (  
     `id` int(11) DEFAULT NULL,  
     `name` varchar(20) DEFAULT NULL,  
     `order_time` datetime NOT NULL,  
      PRIMARY KEY (`order_time`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
   /*!50500 PARTITION BY RANGE  COLUMNS(order_time) */ 
   /*!99990 800020200 INTERVAL(YEAR, 1) */
   /*!50500 (PARTITION p20200101000000 VALUES LESS THAN ('2020-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION p20210101000000 VALUES LESS THAN ('2021-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION p20220101000000 VALUES LESS THAN ('2022-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20230101000000 VALUES LESS THAN ('2023-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20240101000000 VALUES LESS THAN ('2024-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20250101000000 VALUES LESS THAN ('2025-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20260101000000 VALUES LESS THAN ('2026-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20270101000000 VALUES LESS THAN ('2027-01-01 00:00:00') ENGINE = InnoDB, 
   PARTITION _p20280101000000 VALUES LESS THAN ('2028-01-01 00:00:00') ENGINE = InnoDB) */
   1 row in set (0.03 sec)

   已經(jīng)創(chuàng)建了新的分區(qū)，且分區(qū)數(shù)量大于3，滿足DLM策略的執(zhí)行條件，可以進行數(shù)據(jù)歸檔。

創(chuàng)建DLM策略

以下示例將創(chuàng)建一張表名為sales的分區(qū)表，該表以order_time列作為分區(qū)鍵，按時間間隔劃分分區(qū)。同時，該表存在INTERVAL和DLM兩種策略：

• INTERVAL策略：當插入的數(shù)據(jù)超過分區(qū)范圍時，將自動創(chuàng)建新的分區(qū)，時間間隔為1年。

• DLM策略：定義當前表僅有3個分區(qū)，當分區(qū)數(shù)量大于3時，執(zhí)行DLM策略會直接將之前的分區(qū)轉(zhuǎn)存至OSS。

1. 創(chuàng)建sales表。

   CREATE TABLE `sales` (
     `id` int DEFAULT NULL,
     `name` varchar(20) DEFAULT NULL,
     `order_time` datetime NOT NULL,
      PRIMARY KEY (`order_time`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
   PARTITION BY RANGE  COLUMNS(order_time) INTERVAL(YEAR, 1)
   (PARTITION p20200101000000 VALUES LESS THAN ('2020-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20210101000000 VALUES LESS THAN ('2021-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20220101000000 VALUES LESS THAN ('2022-01-01 00:00:00') ENGINE = InnoDB)
   DLM ADD POLICY policy_part2part TIER TO PARTITION ENGINE=CSV STORAGE=OSS READ ONLY ON (PARTITIONS OVER 3);

   該表的DLM策略名稱為policy_part2part，當分區(qū)數(shù)量大于3時，會將舊的分區(qū)轉(zhuǎn)存至OSS。

2. 在mysql.dlm_policies表中查看DLM策略。

   SELECT * FROM mysql.dlm_policies\G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************
                      Id: 2
            Table_schema: test
              Table_name: sales
             Policy_name: policy_part2part
             Policy_type: PARTITION
            Archive_type: PARTITION COUNT
            Storage_mode: READ ONLY
          Storage_engine: CSV
           Storage_media: OSS
     Storage_schema_name: NULL
      Storage_table_name: NULL
         Data_compressed: OFF
    Compressed_algorithm: NULL
                 Enabled: ENABLED
         Priority_number: 10300
   Tier_partition_number: 3
          Tier_condition: NULL
              Extra_info: null
                 Comment: NULL
   1 row in set (0.03 sec)

3. 使用proc_batch_insert存儲過程向sales分區(qū)表中插入一定的測試數(shù)據(jù)，以觸發(fā)INTERVAL策略來自動創(chuàng)建新的分區(qū)。

   CALL proc_batch_insert(1, 3000, 'sales');

   執(zhí)行結(jié)果如下，表示數(shù)據(jù)插入成功：

   Query OK, 1 row affected, 1 warning (0.99 sec)

4. 執(zhí)行以下命令，查看sales表的結(jié)構信息。

   SHOW CREATE TABLE sales \G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************
          Table: sales
   Create Table: CREATE TABLE `sales` (
     `id` int(11) DEFAULT NULL,
     `name` varchar(20) DEFAULT NULL,
     `order_time` datetime DEFAULT NULL,
      PRIMARY KEY (`order_time`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
   /*!50500 PARTITION BY RANGE  COLUMNS(order_time) */ /*!99990 800020200 INTERVAL(YEAR, 1) */
   /*!50500 (PARTITION p20200101000000 VALUES LESS THAN ('2020-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20210101000000 VALUES LESS THAN ('2021-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION p20220101000000 VALUES LESS THAN ('2022-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20230101000000 VALUES LESS THAN ('2023-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20240101000000 VALUES LESS THAN ('2024-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20250101000000 VALUES LESS THAN ('2025-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20260101000000 VALUES LESS THAN ('2026-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20270101000000 VALUES LESS THAN ('2027-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20280101000000 VALUES LESS THAN ('2028-01-01 00:00:00') ENGINE = InnoDB) */
   1 row in set (0.03 sec)

執(zhí)行DLM策略

1. 通過以下命令來執(zhí)行DLM策略。

   CALL dbms_dlm.execute_all_dlm_policies();

2. 通過mysql.dlm_progress表查看DLM執(zhí)行記錄。

   SELECT * FROM mysql.dlm_progress \G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************
                     Id: 4
           Table_schema: test
             Table_name: sales
            Policy_name: policy_part2part
            Policy_type: PARTITION
         Archive_option: PARTITIONS OVER 3
         Storage_engine: CSV
          Storage_media: OSS
        Data_compressed: OFF
   Compressed_algorithm: NULL
     Archive_partitions: p20200101000000, p20210101000000, p20220101000000, _p20230101000000, _p20240101000000, _p20250101000000
          Archive_stage: ARCHIVE_COMPLETE
     Archive_percentage: 100
     Archived_file_info: null
             Start_time: 2023-09-11 18:04:39
               End_time: 2023-09-11 18:04:40
             Extra_info: null
   1 row in set (0.02 sec)

3. 執(zhí)行以下命令，查看sales表結(jié)構信息。

   SHOW CREATE TABLE sales \G

   執(zhí)行結(jié)果如下：

   *************************** 1. row ***************************
          Table: sales
   Create Table: CREATE TABLE `sales` (
     `id` int(11) DEFAULT NULL,
     `name` varchar(20) DEFAULT NULL,
     `order_time` datetime DEFAULT NULL,
      PRIMARY KEY (`order_time`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci CONNECTION='default_oss_server'
   /*!99990 800020205 PARTITION BY RANGE  COLUMNS(order_time) */ /*!99990 800020200 INTERVAL(YEAR, 1) */
   /*!99990 800020205 (PARTITION p20200101000000 VALUES LESS THAN ('2020-01-01 00:00:00') ENGINE = CSV,
    PARTITION p20210101000000 VALUES LESS THAN ('2021-01-01 00:00:00') ENGINE = CSV,
    PARTITION p20220101000000 VALUES LESS THAN ('2022-01-01 00:00:00') ENGINE = CSV,
    PARTITION _p20230101000000 VALUES LESS THAN ('2023-01-01 00:00:00') ENGINE = CSV,
    PARTITION _p20240101000000 VALUES LESS THAN ('2024-01-01 00:00:00') ENGINE = CSV,
    PARTITION _p20250101000000 VALUES LESS THAN ('2025-01-01 00:00:00') ENGINE = CSV,
    PARTITION _p20260101000000 VALUES LESS THAN ('2026-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20270101000000 VALUES LESS THAN ('2027-01-01 00:00:00') ENGINE = InnoDB,
    PARTITION _p20280101000000 VALUES LESS THAN ('2028-01-01 00:00:00') ENGINE = InnoDB) */
   1 row in set (0.03 sec)

   由查詢到的表結(jié)構信息可以看出，sales分區(qū)表中的p20200101000000、p20210101000000、p20220101000000、_p20230101000000、_p20240101000000以及_p20250101000000分區(qū)均轉(zhuǎn)存至OSS，InnoDB引擎中僅保留了3個熱數(shù)據(jù)分區(qū)_p20260101000000、_p20270101000000和_p20280101000000。sales表變?yōu)榱嘶旌戏謪^(qū)表，查詢混合分區(qū)表中的數(shù)據(jù)操作方法請參見查詢混合分區(qū)。