Create a Google Cloud Dataproc cluster.

yaml
type: "io.kestra.plugin.gcp.dataproc.clusters.create"

Creates a cluster in Google Cloud Dataproc.

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: cluster_create
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    workerMachineType: n1-standard-2
    workers: 2
    bucket: YOUR_BUCKET_NAME

Creates a cluster in Google Cloud Dataproc with specific disk size.

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: create_cluster_with_certain_disk_size
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    masterDiskSizeGB: 500
    workerMachineType: n1-standard-2
    workerDiskSizeGB: 200
    workers: 2
    bucket: YOUR_BUCKET_NAM

Dynamic YES

Min length 1

The cluster name.

Dynamic YES

Min length 1

The region.

Dynamic YES

The GCS bucket name.

Dynamic YES

The Dataproc image URI.

The Compute Engine image resource used for cluster instances.

Dynamic YES

The GCP service account to impersonate.

Dynamic NO

The disk size in GB for each master node.

Dynamic YES

The master machine type.

Dynamic YES

The GCP project ID.

SubType string

Dynamic YES

Default ["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used.

Dynamic YES

The GCP service account.

Dynamic NO

The disk size in GB for each worker node.

Dynamic YES

The worker machine type.

Dynamic NO

The number of workers.

Dynamic YES

The zone.

The cluster name.

Default false

Whether cluster was created successfully.