Struct google_api_proto::google::dataflow::v1beta3::Environment

pub struct Environment {

    pub temp_storage_prefix: String,
    pub cluster_manager_api_service: String,
    pub experiments: Vec<String>,
    pub service_options: Vec<String>,
    pub service_kms_key_name: String,
    pub worker_pools: Vec<WorkerPool>,
    pub user_agent: Option<Struct>,
    pub version: Option<Struct>,
    pub dataset: String,
    pub sdk_pipeline_options: Option<Struct>,
    pub internal_experiments: Option<Any>,
    pub service_account_email: String,
    pub flex_resource_scheduling_goal: i32,
    pub worker_region: String,
    pub worker_zone: String,
    pub shuffle_mode: i32,
    pub debug_options: Option<DebugOptions>,
}

Describes the environment in which a Dataflow Job runs.

Fields

temp_storage_prefix: String

The prefix of the resources the system should use for temporary storage. The system will append the suffix “/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

cluster_manager_api_service: String

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. “compute.googleapis.com”.

experiments: Vec<String>

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

service_options: Vec<String>

The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

service_kms_key_name: String

If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

worker_pools: Vec<WorkerPool>

The worker pools. At least one “harness” worker pool must be specified in order for the job to have workers.

user_agent: Option<Struct>

A description of the process that generated the request.

version: Option<Struct>

A structure describing which components and their versions of the service are required in order to run the job.

dataset: String

The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}

sdk_pipeline_options: Option<Struct>

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

internal_experiments: Option<Any>

Experimental settings.

service_account_email: String

Identity to run virtual machines as. Defaults to the default account.

flex_resource_scheduling_goal: i32

Which Flexible Resource Scheduling mode to run in.

worker_region: String

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. “us-west1”. Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane’s region.

worker_zone: String

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. “us-west1-a”. Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane’s region is chosen based on available capacity.

shuffle_mode: i32

Output only. The shuffle mode used for the job.

debug_options: Option<DebugOptions>

Any debugging options to be supplied to the job.

Implementations

impl Environment

pub fn flex_resource_scheduling_goal(&self) -> FlexResourceSchedulingGoal

Returns the enum value of flex_resource_scheduling_goal, or the default if the field is set to an invalid enum value.

pub fn set_flex_resource_scheduling_goal( &mut self, value: FlexResourceSchedulingGoal, )

Sets flex_resource_scheduling_goal to the provided enum value.

pub fn shuffle_mode(&self) -> ShuffleMode

Returns the enum value of shuffle_mode, or the default if the field is set to an invalid enum value.

pub fn set_shuffle_mode(&mut self, value: ShuffleMode)

Sets shuffle_mode to the provided enum value.

Trait Implementations

impl Clone for Environment

fn clone(&self) -> Environment

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Environment

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Environment

fn default() -> Self

Returns the “default value” for a type.

impl Message for Environment

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl PartialEq for Environment

fn eq(&self, other: &Environment) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Environment