Struct google_api_proto::google::cloud::bigquery::v2::ExternalDataConfiguration

pub struct ExternalDataConfiguration {

    pub source_uris: Vec<String>,
    pub file_set_spec_type: i32,
    pub schema: Option<TableSchema>,
    pub source_format: String,
    pub max_bad_records: Option<i32>,
    pub autodetect: Option<bool>,
    pub ignore_unknown_values: Option<bool>,
    pub compression: String,
    pub csv_options: Option<CsvOptions>,
    pub json_options: Option<JsonOptions>,
    pub bigtable_options: Option<BigtableOptions>,
    pub google_sheets_options: Option<GoogleSheetsOptions>,
    pub hive_partitioning_options: Option<HivePartitioningOptions>,
    pub connection_id: String,
    pub decimal_target_types: Vec<i32>,
    pub avro_options: Option<AvroOptions>,
    pub json_extension: i32,
    pub parquet_options: Option<ParquetOptions>,
    pub object_metadata: Option<i32>,
    pub reference_file_schema_uri: Option<String>,
    pub metadata_cache_mode: i32,
}

Fields

source_uris: Vec<String>

[Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the ‘’ wildcard character is not allowed.

file_set_spec_type: i32

Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.

schema: Option<TableSchema>

Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.

source_format: String

[Required] The data format. For CSV files, specify “CSV”. For Google sheets, specify “GOOGLE_SHEETS”. For newline-delimited JSON, specify “NEWLINE_DELIMITED_JSON”. For Avro files, specify “AVRO”. For Google Cloud Datastore backups, specify “DATASTORE_BACKUP”. For Apache Iceberg tables, specify “ICEBERG”. For ORC files, specify “ORC”. For Parquet files, specify “PARQUET”. [Beta] For Google Cloud Bigtable, specify “BIGTABLE”.

max_bad_records: Option<i32>

Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.

autodetect: Option<bool>

Try to detect schema and format options automatically. Any option specified explicitly will be honored.

ignore_unknown_values: Option<bool>

Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don’t match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.

compression: String

Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.

csv_options: Option<CsvOptions>

Optional. Additional properties to set if sourceFormat is set to CSV.

json_options: Option<JsonOptions>

Optional. Additional properties to set if sourceFormat is set to JSON.

bigtable_options: Option<BigtableOptions>

Optional. Additional options if sourceFormat is set to BIGTABLE.

google_sheets_options: Option<GoogleSheetsOptions>

Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.

hive_partitioning_options: Option<HivePartitioningOptions>

Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning – requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.

connection_id: String

Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form {project_id}.{location_id};{connection_id} or projects/{project_id}/locations/{location_id}/connections/{connection_id}.

decimal_target_types: Vec<i32>

Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown.

Example: Suppose the value of this field is [“NUMERIC”, “BIGNUMERIC”]. If (precision,scale) is:

• (38,9) -> NUMERIC;
• (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits);
• (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits);
• (76,38) -> BIGNUMERIC;
• (77,38) -> BIGNUMERIC (error if value exeeds supported range).

This field cannot contain duplicate types. The order of the types in this field is ignored. For example, [“BIGNUMERIC”, “NUMERIC”] is the same as [“NUMERIC”, “BIGNUMERIC”] and NUMERIC always takes precedence over BIGNUMERIC.

Defaults to [“NUMERIC”, “STRING”] for ORC and [“NUMERIC”] for the other file formats.

avro_options: Option<AvroOptions>

Optional. Additional properties to set if sourceFormat is set to AVRO.

json_extension: i32

Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).

parquet_options: Option<ParquetOptions>

Optional. Additional properties to set if sourceFormat is set to PARQUET.

object_metadata: Option<i32>

Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted.

Currently SIMPLE is the only supported Object Metadata type.

reference_file_schema_uri: Option<String>

Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.

metadata_cache_mode: i32

Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.

Implementations

impl ExternalDataConfiguration

pub fn decimal_target_types( &self, ) -> FilterMap<Cloned<Iter<'_, i32>>, fn(_: i32) -> Option<DecimalTargetType>>

Returns an iterator which yields the valid enum values contained in decimal_target_types.

pub fn push_decimal_target_types(&mut self, value: DecimalTargetType)

Appends the provided enum value to decimal_target_types.

pub fn json_extension(&self) -> JsonExtension

Returns the enum value of json_extension, or the default if the field is set to an invalid enum value.

pub fn set_json_extension(&mut self, value: JsonExtension)

Sets json_extension to the provided enum value.

pub fn object_metadata(&self) -> ObjectMetadata

Returns the enum value of object_metadata, or the default if the field is unset or set to an invalid enum value.

pub fn set_object_metadata(&mut self, value: ObjectMetadata)

Sets object_metadata to the provided enum value.

pub fn metadata_cache_mode(&self) -> MetadataCacheMode

Returns the enum value of metadata_cache_mode, or the default if the field is set to an invalid enum value.

pub fn set_metadata_cache_mode(&mut self, value: MetadataCacheMode)

Sets metadata_cache_mode to the provided enum value.

pub fn file_set_spec_type(&self) -> FileSetSpecType

Returns the enum value of file_set_spec_type, or the default if the field is set to an invalid enum value.

pub fn set_file_set_spec_type(&mut self, value: FileSetSpecType)

Sets file_set_spec_type to the provided enum value.

Trait Implementations

impl Clone for ExternalDataConfiguration

fn clone(&self) -> ExternalDataConfiguration

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ExternalDataConfiguration

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ExternalDataConfiguration

fn default() -> Self

Returns the “default value” for a type.

impl Message for ExternalDataConfiguration

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl PartialEq for ExternalDataConfiguration

fn eq(&self, other: &ExternalDataConfiguration) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ExternalDataConfiguration