前提條件

已創(chuàng)建包含Spark和Hudi服務(wù)的集群，詳情請(qǐng)參見(jiàn)創(chuàng)建集群。

使用限制

EMR-3.36.0及后續(xù)版本和EMR-5.2.0及后續(xù)版本，支持Spark SQL對(duì)Hudi進(jìn)行讀寫操作。

啟動(dòng)方式

• Spark2和Spark3 hudi0.11以下版本

  spark-sql \
  --conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
  --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'

• Spark3 hudi0.11及以上版本

  spark-sql \
  --conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
  --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' \
  --conf 'spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog'

MERGE INTO

表示根據(jù)匹配條件執(zhí)行插入、更新或刪除操作。

• 語(yǔ)法

  MERGE INTO tableIdentifier AS target_alias
  USING (sub_query | tableIdentifier) AS source_alias
  ON <merge_condition>
   WHEN MATCHED [ AND <condition> ] THEN <matched_action> 
  [ WHEN MATCHED [ AND <condition> ] THEN <matched_action> ]
  [ WHEN NOT MATCHED [ AND <condition> ]  THEN <not_matched_action> ]
  
  <merge_condition> =A equal bool condition 
  <matched_action>  =
    DELETE  |
    UPDATE SET *  |
    UPDATE SET column1 = value1 [, column2 = value2 ...]
  <not_matched_action>  =
    INSERT *  |
    INSERT (column1 [, column2 ...]) VALUES (value1 [, value2 ...])

• 示例

  -- without delete
  merge into h0 as target
  using (
    select 1 as id, 'a1' as name, 10.0 as price
  ) source
  on target.id = source.id
  when matched then update set *
  when not matched then insert *;
  -- with delete
  merge into h0 as target
  using (
    select 1 as id, 'a1' as name, 10.0 as price
  ) source
  on target.id = source.id
  when matched then update set id = source.id, name = source.name, price = source.price
  when matched and name = 'delete' then delete
  when not matched then insert (id,name,price) values(id, name, price);

INSERT INTO

表示向分區(qū)表或非分區(qū)表插入數(shù)據(jù)。

UPDATE語(yǔ)句

表示更新分區(qū)表或非分區(qū)表中行對(duì)應(yīng)的單列或多列數(shù)據(jù)。

• 語(yǔ)法

  UPDATE tableIdentifier SET column = EXPRESSION(,column = EXPRESSION);

• 示例：將表h0中id為1的price字段值更新為20。

  update h0 set price=20 where id=1;

DELETE語(yǔ)句

表示刪除分區(qū)表或非分區(qū)表中滿足指定條件的單行或多行數(shù)據(jù)。

• 語(yǔ)法

  DELETE FROM tableIdentifier [WHERE BOOL_EXPRESSION];

• 示例：刪除表h0中id大于100的數(shù)據(jù)。

  delete from h0 where id>100;