每個字典元素包含的key如下表所示。

在O1優化級別中，提供test_data可以使優化更具指向性。在O2優化級別中，提供calibration_dataset可以指導Blade進行離線INT8優化。這兩個參數均需符合Blade的優化輔助數據格式。TensorFlow和PyTorch的優化輔助數據格式如下所示：

• TensorFlow的輔助數據是List of Feed Dict格式。Feed Dict的key是字符串，value是numpy的ndarray。輔助數據文件保存為 .npy文件。

• PyTorch的輔助數據是List of Tensor Tuple格式，保存為.pth文件。

您可以參考如下示例代碼生成TensorFlow的輔助數據文件。

import numpy as np

calib_data = list()
for i in range(10):
    feed_dict = {
        'image_placeholder:0': np.ones((8, 3, 224, 224), dtype=np.float32),
        'threshold_placeholder:0': np.float32(0.5),
    }
    calib_data.append(feed_dict)
np.save("calib_data.npy", calib_data)

您可以參考如下示例代碼生成PyTorch的輔助數據文件。

import numpy as np
import torch

calib_data = list()
for i in range(10):
    image = torch.ones(8, 3, 224, 224)
    threshold = torch.tensor(0.5)
    feed_tuple = (image, threshold)
    calib_data.append(feed_tuple)
torch.save(calib_data, 'calib_data.pth')

如下是一個簡單的、不使用Blade EAS Plugin的EAS服務配置文件。

{
  "name": "blade_eas_plugin_test",
  "model_path": "oss://<yourBucket>/<pathToYourModel>/",
  "processor": "tensorflow_cpu_1.15",
  "metadata": {
    "instance": 1,
    "memory": 4000
  },
  "resource": "<yourEASResource>"
}

關于EAS服務配置文件的字段詳情，請參見創建服務。

將以上服務配置文件保存為service.json，使用如下命令即可創建一個部署在CPU上的TensorFlow 1.15服務。

eascmd create service.json

系統輸出類似如下結果。

+-------------------+-------------------------------------------------------------------------------------------------+
| Internet Endpoint | http://123456789012****.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test       |
| Intranet Endpoint | http://123456789012****.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test   |
|             Token | owic823JI3kRmMDZlOTQzMTA3ODhmOWIzMmVkZmZmZGQyNmQ1N2M5****                                        |
+-------------------+-------------------------------------------------------------------------------------------------+
[OK] Service is now creating
[OK] Schedule process to node cn-shanghai.i-uf6hv6kfua25q1k8****
[OK] Fetching processor from [http://pai-blade.oss-cn-zhangjiakou.aliyuncs.com/release/3.18.0/py3.6.8_cpu_tf1.15.0_torch1.6.0_abiprecxx11/TENSORFLOW_SDK_CPU.d12d3dc-91024d0-1.15.0-Linux.tar.gz]
[OK] Successfully downloaded all artifacts
[OK] Building image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117170541
[OK] Pushing image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117170541
[OK] Successfully pushed image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117170541
[OK] Successfully created ingress
[OK] Successfully synchronized resources
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Running [Total: 1, Pending: 0, Running: 1]
[OK] Service is running

啟動Blade EAS Plugin，您只需要在EAS服務配置文件中增加plugins字段。該字段是一個列表，在如下示例中只有一個Blade的優化插件，所以列表中只有一個字典元素。

{
  "name": "blade_eas_plugin_test",
  "model_path": "oss://<yourBucket>/<pathToYourModel>/",
  "processor": "tensorflow_cpu_1.15",
  "metadata": {
    "instance": 1,
    "memory": 4000
  },
  "plugins": [
      {
          "command": "blade --mode eas_plugin --optimize_for cpu",
          "image": "registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest"
      }
  ],
  "resource": "<yourEASResource>"
}

上述示例中的除 plugins外，其他字段仍然遵循PAI-EAS服務配置文件的格式，詳情請參見創建服務。plugins列表下的字典元素包含如下兩個key：

• command字段：表示運行的優化命令。其中--mode eas_plugin表示執行EAS Plugin優化模式，--optimize_for cpu表示為CPU推理進行優化。

• image字段：表示Blade EAS Plugin的鏡像地址，所有CPU優化都使用鏡像registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest。

上述示例即完成了CPU設備上不帶測試數據的優化配置。將以上服務配置文件保存為service1.json，通過eascmd客戶端工具的create命令即可創建服務，如下所示。

eascmd create service1.json

系統輸出類似如下結果。

+-------------------+-------------------------------------------------------------------------------------------------+
| Internet Endpoint | http://123456789012****.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test       |
| Intranet Endpoint | http://123456789012****.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test   |
|             Token | owic823JI3kRmMDZlOTQzMTA3ODhmOWIzMmVkZmZmZGQyNmQ1N2M5****                                       |
+-------------------+-------------------------------------------------------------------------------------------------+
[OK] Service is now creating
[OK] Fetching processor from [http://pai-blade.oss-cn-zhangjiakou.aliyuncs.com/release/3.18.0/py3.6.8_cpu_tf1.15.0_torch1.6.0_abiprecxx11/TENSORFLOW_SDK_CPU.d12d3dc-91024d0-1.15.0-Linux.tar.gz]
[OK] Successfully downloaded all artifacts
[OK] Executing plugin eas-plugin-73d70d54: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest
[OK] Building image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117172259
[OK] Pushing image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117172259
[OK] Successfully pushed image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117172259
[OK] Successfully created ingress
[OK] Successfully patch resources
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Running [Total: 1, Pending: 0, Running: 1]
[OK] Service is running

觀察上述日志，您可以發現對比不啟動Blade EAS Plugin的配置，多了如下一行日志，表示成功執行了Blade優化。

[OK] Executing plugin eas-plugin-73d70d54: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest

通常情況下，提供更多有關模型的信息有助于提升優化的準確性和加速比。以下示例帶有更多優化參數，且以GPU為例，服務描述文件如下所示。

{
    "name": "blade_eas_plugin_test",
    "metadata": {
        "cpu": 4,
        "gpu": 1,
        "instance": 1,
        "memory": 4096,
        "cuda": "10.0"
    },
    "model_path": "oss://<yourBucket>/<pathToYourModel>/",
    "plugins": [
        {
            "command": "blade --mode eas_plugin --optimize_for gpu",
            "image": "registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest",
            "resource": "T4_8CORE",
            "gpu": 1,
            "config": {
                "model_info": {
                    "frozen.pb": {
                        "optimization_level": "o1",
                        "inputs": [
                            "input_ids_a_1"
                        ],
                        "outputs": [
                            "l2_normalize"
                        ],
                        "test_data": "test_len9240_bc1.npy"
                    }
                }
            }
        }
    ],
    "processor": "tensorflow_gpu_1.15",
    "resource": "<yourEASResource>"
}

上述示例中除 plugins外，其他字段仍然遵循EAS服務配置文件的格式，詳情請參見創建服務。plugins包含的字段詳情請參見上文的plugins包含的字段。此外，上述服務配置文件中的frozen.pb是模型的文件名，表示優化fronzon.pb文件中的TensorFlow模型。

將以上服務配置文件保存為service2.json，通過eascmd客戶端工具的create命令即可創建服務，如下所示。

eascmd create service2.json

系統輸出類似如下結果。

+-------------------+-------------------------------------------------------------------------------------------------+
| Internet Endpoint | http://123456789012****cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test      |
| Intranet Endpoint | http://123456789012****.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test  |
|             Token | owic823JI3kRmMDZlOTQzMTA3ODhmOWIzMmVkZmZmZGQyNmQ1N2M5****                                        |
+-------------------+-------------------------------------------------------------------------------------------------+
[OK] Service is now creating
[OK] Schedule process to node cn-shanghai.i-uf642ocg20xinsme****
[OK] Downloading oss file: oss://blade-qa/test_assets/tf_security_textcnn/
[OK] Fetching processor from [http://pai-blade.oss-cn-zhangjiakou.aliyuncs.com/release/3.18.0/py3.6.8_cu100_tf1.15.0_torch1.6.0_abiprecxx11/TENSORFLOW_SDK_GPU.d12d3dc-91024d0-1.15.0-Linux.tar.gz]
[OK] Successfully downloaded all artifacts
[OK] Executing plugin eas-plugin-7126ee68: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest
[OK] Building image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117191732
[OK] Pushing image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117191732
[OK] Successfully pushed image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117191732
[OK] Successfully created ingress
[OK] Successfully synchronized resources
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Running [Total: 1, Pending: 0, Running: 1]
[OK] Service is running

觀察上述日志，您可以發現對比不啟動Blade EAS Plugin的配置，多了如下一行日志，表示成功執行了Blade優化。

[OK] Executing plugin eas-plugin-7126ee68: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest