CreateTable

Create a table

yaml
type: "io.kestra.plugin.gcp.bigquery.CreateTable"

yaml
id: gcp_bq_create_table
namespace: company.team

tasks:
  - id: create_table
    type: io.kestra.plugin.gcp.bigquery.CreateTable
    projectId: my-project
    dataset: my-dataset
    table: my-table
    tableDefinition:
      type: TABLE
      schema:
        fields:
        - name: id
          type: INT64
        - name: name
          type: STRING
      standardTableDefinition:
        clustering:
        - id
        - name
    friendlyName: new_table

Dynamic YES

The dataset's user-defined ID.

Dynamic YES

The table's user-defined ID.

Dynamic YES

The user-friendly description for the table.

Dynamic NO

The encryption configuration.

Dynamic YES

Format duration

Sets the duration, since now, when this table expires.

If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

Dynamic YES

The user-friendly name for the table.

Dynamic YES

The GCP service account to impersonate.

SubType string

Dynamic YES

Return a map for labels applied to the table.

Dynamic YES

The geographic location where the dataset should reside.

This property is experimental and might be subject to change or removed.

See Dataset Location

Dynamic YES

The GCP project ID.

Dynamic YES

Return true if a partition filter (that can be used for partition elimination) is required for queries over this table.

Dynamic NO

Automatic retry for retryable BigQuery exceptions.

Some exceptions (especially rate limit) are not retried by default by BigQuery client, we use by default a transparent retry (not the kestra one) to handle this case. The default values are exponential of 5 seconds for a maximum of 15 minutes and ten attempts

SubType string

Dynamic YES

Default ["due to concurrent update","Retrying the job may solve the problem"]

The messages which would trigger an automatic retry.

Message is tested as a substring of the full message, and is case insensitive.

SubType string

Dynamic YES

Default ["rateLimitExceeded","jobBackendError","internalError","jobInternalError"]

The reasons which would trigger an automatic retry.

SubType string

Dynamic YES

Default ["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used.

Dynamic YES

The GCP service account.

Dynamic NO

The table definition.

Format date-time

The time when this table was created.

The dataset's ID.

The table definition.

The user-friendly description for the table.

The encryption configuration.

The hash of the table resource.

Format date-time

Returns the time when this table expires.

If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

The user-friendly name for the table.

The service-generated id for the table.

SubType string

Return a map for labels applied to the table.

Format date-time

The time when this table was last modified.

The size of this table in bytes.

The number of bytes considered "long-term storage" for reduced billing purposes.

The number of rows of data in this table.

The project's ID.

Return true if a partition filter (that can be used for partition elimination) is required for queries over this table.

The URL that can be used to access the resource again. The returned URL can be used for get or update requests.

The table name.

Dynamic NO

Format duration

Dynamic NO

Default RETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

Dynamic NO

Minimum >= 1

Dynamic NO

Format duration

Dynamic NO

Default constant

Dynamic NO

Default false

Dynamic NO

The external table definition if the type is EXTERNAL.

Dynamic NO

The materialized view definition if the type is MATERIALIZED_VIEW.

Dynamic NO

The table's schema.

Dynamic NO

The table definition if the type is TABLE.

Dynamic NO

The view definition if the type is VIEW.

Dynamic NO

SubType string

Dynamic YES

Returns the clustering configuration for this table. If {@code null}, the table is not clustered.

Dynamic NO

Returns the range partitioning configuration for this table. If {@code null}, the table is not range-partitioned.

Dynamic NO

Returns information on the table's streaming buffer, if exists. Returns {@code null} if no streaming buffer exists.

Dynamic NO

Returns the time partitioning configuration for this table. If {@code null}, the table is not time-partitioned.

Dynamic NO

Format duration

Dynamic NO

Format duration

Dynamic NO

Default RETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

Dynamic NO

Minimum >= 1

Dynamic NO

Format duration

Dynamic NO

Default exponential

Dynamic NO

Default false

Dynamic NO

SubType

Dynamic NO

Dynamic YES

Dynamic NO

Format duration

Dynamic NO

Format duration

Dynamic NO

Default RETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

Dynamic NO

Minimum >= 1

Dynamic NO

Format duration

Dynamic NO

Default random

Dynamic NO

Default false

Dynamic NO

Possible Values

TABLEVIEWMATERIALIZED_VIEWEXTERNALMODEL

Dynamic NO

SubType string

Dynamic YES

Dynamic NO

Dynamic YES

Dynamic NO

Format date-time

Dynamic NO

Format duration

SubType

Dynamic NO

Dynamic YES

The query whose result is persisted.

SubType

Dynamic NO

User defined functions that can be used by query. Returns {@code null} if not set.

Dynamic NO

Dynamic YES

SubType string

Dynamic NO

Dynamic YES

Whether automatic refresh is enabled for the materialized view when the base table is updated.

Dynamic NO

Format date-time

Date when this materialized view was last modified.

Dynamic YES

The query whose result is persisted.

Dynamic YES

Format duration

The maximum frequency at which this materialized view will be refreshed.

Dynamic YES

Format duration

The number of milliseconds for which to keep the storage for a partition. When expired, the storage for the partition is reclaimed. If null, the partition does not expire.

Dynamic YES

If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field.

Dynamic YES

If set to true, queries over this table require a partition filter (that can be used for partition elimination) to be specified.

Dynamic YES

The field description.

Dynamic YES

Possible Values

NULLABLEREQUIREDREPEATED

The field mode.

By default, Field.Mode.NULLABLE is used.

Dynamic YES

The field name.

Dynamic NO

The policy tags for the field.

SubType

Dynamic NO

The list of sub-fields if type is a LegacySQLType.RECORD. Returns null otherwise.

Dynamic YES

Possible Values

INLINEFROM_URI

Dynamic YES

If type is UserDefinedFunction.Type.INLINE, this method returns a code blob. If type is UserDefinedFunction.Type.FROM_URI, the method returns a Google Cloud Storage URI (e.g. gs://bucket/path).

Dynamic YES

Whether automatic detection of schema and format options should be performed.

Dynamic YES

The compression type of the data source.

Dynamic YES

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

The source format, and possibly some parsing options, of the external data.

Dynamic YES

Whether BigQuery should allow extra values that are not represented in the table schema.

If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result.

Dynamic YES

The maximum number of bad records that BigQuery can ignore when reading data.

If the number of bad records exceeds this value, an invalid error is returned in the job result.

SubType string

Dynamic YES

The fully-qualified URIs that point to your data in Google Cloud Storage.

Each URI can

contain one '*' wildcard character that must come after the bucket's name. Size limits related
to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size
across all URIs.

Dynamic NO

Possible Values

NULLABLEREQUIREDREPEATED

Dynamic NO

SubType

Dynamic NO

Possible Values

BOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGE

Dynamic NO

SubType

Dynamic NO

Format duration

Dynamic NO

Possible Values

DAYHOURMONTHYEAR

Dynamic NO

Possible Values

INLINEFROM_URI

Dynamic NO

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

Dynamic NO

SubType string

Dynamic NO

SubType string

Dynamic NO

​Create​Table

CreateTable