Struct google_api_proto::google::dataflow::v1beta3::WorkerPool

pub struct WorkerPool {

    pub kind: String,
    pub num_workers: i32,
    pub packages: Vec<Package>,
    pub default_package_set: i32,
    pub machine_type: String,
    pub teardown_policy: i32,
    pub disk_size_gb: i32,
    pub disk_type: String,
    pub disk_source_image: String,
    pub zone: String,
    pub taskrunner_settings: Option<TaskRunnerSettings>,
    pub on_host_maintenance: String,
    pub data_disks: Vec<Disk>,
    pub metadata: BTreeMap<String, String>,
    pub autoscaling_settings: Option<AutoscalingSettings>,
    pub pool_args: Option<Any>,
    pub network: String,
    pub subnetwork: String,
    pub worker_harness_container_image: String,
    pub num_threads_per_worker: i32,
    pub ip_configuration: i32,
    pub sdk_harness_container_images: Vec<SdkHarnessContainerImage>,
}

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Fields

kind: String

The kind of the worker pool; currently only harness and shuffle are supported.

num_workers: i32

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

packages: Vec<Package>

Packages to be installed on workers.

default_package_set: i32

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

machine_type: String

Machine type (e.g. “n1-standard-1”). If empty or unspecified, the service will attempt to choose a reasonable default.

teardown_policy: i32

Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user’s project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

If unknown or unspecified, the service will attempt to choose a reasonable default.

disk_size_gb: i32

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

disk_type: String

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

disk_source_image: String

Fully qualified source image for disks.

zone: String

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

taskrunner_settings: Option<TaskRunnerSettings>

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

on_host_maintenance: String

The action to take on host maintenance, as defined by the Google Compute Engine API.

data_disks: Vec<Disk>

Data disks that are used by a VM in this workflow.

metadata: BTreeMap<String, String>

Metadata to set on the Google Compute Engine VMs.

autoscaling_settings: Option<AutoscalingSettings>

Settings for autoscaling of this WorkerPool.

pool_args: Option<Any>

Extra arguments for this worker pool.

network: String

Network to which VMs will be assigned. If empty or unspecified, the service will use the network “default”.

subnetwork: String

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form “regions/REGION/subnetworks/SUBNETWORK”.

worker_harness_container_image: String

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

num_threads_per_worker: i32

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

ip_configuration: i32

Configuration for VM IPs.

sdk_harness_container_images: Vec<SdkHarnessContainerImage>

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Implementations

impl WorkerPool

pub fn default_package_set(&self) -> DefaultPackageSet

Returns the enum value of default_package_set, or the default if the field is set to an invalid enum value.

pub fn set_default_package_set(&mut self, value: DefaultPackageSet)

Sets default_package_set to the provided enum value.

pub fn teardown_policy(&self) -> TeardownPolicy

Returns the enum value of teardown_policy, or the default if the field is set to an invalid enum value.

pub fn set_teardown_policy(&mut self, value: TeardownPolicy)

Sets teardown_policy to the provided enum value.

pub fn ip_configuration(&self) -> WorkerIpAddressConfiguration

Returns the enum value of ip_configuration, or the default if the field is set to an invalid enum value.

pub fn set_ip_configuration(&mut self, value: WorkerIpAddressConfiguration)

Sets ip_configuration to the provided enum value.

Trait Implementations

impl Clone for WorkerPool

fn clone(&self) -> WorkerPool

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WorkerPool

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for WorkerPool

fn default() -> Self

Returns the “default value” for a type.

impl Message for WorkerPool

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl PartialEq for WorkerPool

fn eq(&self, other: &WorkerPool) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for WorkerPool