Create clusters in Google Cloud Dataproc.
yaml
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"Creates a cluster in Google Cloud Dataproc.
yaml
id: gcp_dataproc_cluster_create
namespace: company.team
tasks:
- id: cluster_create
type: io.kestra.plugin.gcp.dataproc.clusters.Create
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
workerMachineType: n1-standard-2
workers: 2
bucket: YOUR_BUCKET_NAME
Creates a cluster in Google Cloud Dataproc with specific disk size.
yaml
id: gcp_dataproc_cluster_create
namespace: company.team
tasks:
- id: create_cluster_with_certain_disk_size
type: io.kestra.plugin.gcp.dataproc.clusters.Create
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
masterDiskSizeGB: 500
workerMachineType: n1-standard-2
workerDiskSizeGB: 200
workers: 2
bucket: YOUR_BUCKET_NAM
Dynamic
YES Min length
1The cluster name.
Dynamic
YES Min length
1The region.
Dynamic
YESThe GCS bucket name.
Dynamic
YESThe Dataproc image URI.
The Compute Engine image resource used for cluster instances.
Dynamic
YESThe GCP service account to impersonate.
Dynamic
YESThe disk size in GB for each master node.
Dynamic
YESThe master machine type.
Dynamic
YESThe GCP project ID.
SubType string
Dynamic
YES Default
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used.
Dynamic
YESThe GCP service account.
Dynamic
YESThe disk size in GB for each worker node.
Dynamic
YESThe worker machine type.
Dynamic
YESThe number of workers.
Dynamic
YESThe zone.
The cluster name.
Default
falseWhether cluster was created successfully.