中文文档

mc batch generate

在 MinIO 版本发生变更: RELEASE.2022-10-08T20-11-00Z or later

语法

mc batch generate 命令为指定的作业类型创建一个基本的 YAML 格式的模板文件。

在 MinIO 创建文件后,使用您喜欢的文本编辑器工具打开它以进行进一步的自定义。 每个批处理文件可以定义一个作业任务定义。

请参阅 作业类型,以获取您可以生成的支持的作业类型。

以下命令将在 myminio 别名的 mybucket 桶上为复制作业创建一个基本的YAML文件。

mc batch generate myminio replicate

该命令具有以下语法:

mc [GLOBALFLAGS] batch generate \
                       ALIAS    \
                       JOBTYPE
  • 方括号 [] 表示可选参数。

  • 参数在同一行表示它们是相互依赖的。

  • 使用管道 | 运算符分隔的参数是相互排斥的。

在终端/壳中运行命令之前,将示例复制到文本编辑器中并按需进行修改。

参数

ALIAS
Required

用于生成YAML模板文件的 别名。 指定的 alias 不会限制您在哪里可以使用生成的文件。

例如:

mc batch generate myminio replicate
JOBTYPE
Required

要生成YAML文件的作业类型。

支持以下值:

全局标记

此命令支持任何 全局标志

示例

为复制作业类型生成 yaml 文件

以下命令生成一个用于复制作业的 YAML 蓝图,并将文件命名为 replicate ,扩展名为 .yaml

mc batch generate alias replicate > replicate.yaml
  • alias 替换为要用于生成 YAML 文件的 MinIO 客户端 alias

  • replicate 替换为要生成 YAML 文件的任务类型。

    mc batch 支持 replicatekeyrotate 作业类型。

S3兼容性

mc 命令行工具是为了与 AWS S3 API 兼容而构建的,并且已经过测试, 以确保在与 MinIO 和 AWS S3 配合使用时,功能和行为符合预期。

MinIO 不对其他 S3 兼容服务提供任何保证,因为它们的 S3 API 实现是 未知的,因此不受支持。尽管 mc 命令 可能 如文档中所 述工作,但任何此类使用都是您自己的风险。

作业类型

mc batch 目前支持以下作业任务类型:

  • replicate

    在两个 MinIO 部署之间复制对象。 它提供了与 bucket replication 类似的 functionality,但作为 batch job,而不是持续扫描功能。

  • keyrotate

    在 MinIO 版本加入: RELEASE.2023-04-07T05-28-58Z

    旋转 MinIO 部署上静止对象的 sse-s3 或 sse-kms 密钥。

  • expire

    在 MinIO 版本加入: RELEASE.2023-12-02T10-51-33Z

    根据与 自动对象过期 类似的语义来过期对象。

replicate

您可以使用以下示例配置作为构建您自己的自定义复制批处理作业的起点:

replicate:
  apiVersion: v1
  # source of the objects to be replicated
  # if source is not the local deployment for the command, provide the endpoint and credentials
  source:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX
    # endpoint: ENDPOINT
    # path: "on|off|auto"
    # credentials:
    #   accessKey: ACCESS_KEY
    #   secretKey: SECRET_KEY
    #   sessionToken: SESSION_TOKEN # Available when rotating credentials are used
    # snowball:
    #   disable: true|false
    #   batch: 100
    #   inmemory: true|false
    #   compress: true|false
    #   smallerThan: 5MiB
    #   skipErrs: true|false
  # target where the objects must be replicated
  # if target is not the local deployment for the command, provide the endpoint and credentials
  target:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX
    # endpoint: ENDPOINT
    # path: "on|off|auto"
    # credentials:
    #   accessKey: ACCESS_KEY
    #   secretKey: SECRET_KEY
    #   sessionToken: SESSION_TOKEN # Available when rotating credentials are used

  # optional flags based filtering criteria
  # for all source objects
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "date" # match objects created after "date"
      createdBefore: "date" # match objects created before "date"

      # tags:
      #   - key: "name"
      #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'

      ## NOTE: metadata filter not supported when "source" is non MinIO.
      # metadata:
      #   - key: "content-type"
      #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'

  notify:
    endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
    token: "Bearer xxxxx" # optional authentication token for the notification endpoint

  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: "500ms" # least amount of delay between each retry

请参阅 复制批处理作业参考 以获取关于每个键更完整的文档说明。

keyrotate

您可以使用以下示例配置作为构建您自己的自定义密钥轮换批处理作业的起点:

keyrotate:
  apiVersion: v1
  bucket: bucket
  prefix: 
  encryption:
    type: sse-kms # valid values are sse-s3 and sse-kms
    
    # The following encryption values only apply for sse-kms type.
    # For sse-s3 key types, MinIO uses the key provided by the MINIO_KMS_KES_KEY_FILE environment variable.
    # The following two values are ignored if type is set to sse-s3.
    key: my-new-keys2 # valid only for sse-kms
    context: <new-kms-key-context> # valid only for sse-kms

  # optional flags based filtering criteria
  flags:
    filter:
      newerThan: "84h" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "80h" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "2023-03-02T15:04:05Z07:00" # match objects created after "date"
      createdBefore: "2023-03-02T15:04:05Z07:00" # match objects created before "date"
      tags:
        - key: "name"
          value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
      metadata:
        - key: "content-type"
          value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
      kmskey: "key-id" # match objects with KMS key-id (applicable only for sse-kms)
  
  # optional entries to add notifications for the job
  notify:
    endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
    token: "Bearer xxxxx" # optional authentication token for the notification endpoint
  
  # optional entries to add retry attempts if the job is interrupted
  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: "500ms" # least amount of delay between each retry

请参阅 键轮换批处理作业参考 以获取关于每个键更完整的文档说明。

expire

您可以使用以下示例配置作为构建您自己的自定义过期批处理作业的起点:

expire:
  apiVersion: v1
  bucket: mybucket # Bucket where this job will expire matching objects from
  prefix: myprefix # (Optional) Prefix under which this job will expire objects matching the rules below.
  rules:
    - type: object  # objects with zero ore more older versions
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 70h # match objects older than this value
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before "date"
      tags:
        - key: name
          value: pick* # match objects with tag 'name', all values starting with 'pick'
      metadata:
        - key: content-type
          value: image/* # match objects with 'content-type', all values starting with 'image/'
      size:
        lessThan: 10MiB # match objects with size less than this value (e.g. 10MiB)
        greaterThan: 1MiB # match objects with size greater than this value (e.g. 1MiB)
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object.

    - type: deleted # objects with delete marker as their latest version
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 10h # match objects older than this value (e.g. 7d10h31s)
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before "date"
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object including delete markers.

  notify:
    endpoint: https://notify.endpoint # notification endpoint to receive job completion status
    token: Bearer xxxxx # optional authentication token for the notification endpoint

  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: 500ms # least amount of delay between each retry

请参阅 过期批次处理工作参考 以获取关于每个键更完整的文档说明。

Join Slack 商业支持购买咨询