Module google_api_proto::google::cloud::dataproc::v1

Modules

• autoscaling_policy
  Nested message and enum types in AutoscalingPolicy.
• autoscaling_policy_service_client
  Generated client implementations.
• basic_autoscaling_algorithm
  Nested message and enum types in BasicAutoscalingAlgorithm.
• batch
  Nested message and enum types in Batch.
• batch_controller_client
  Generated client implementations.
• batch_operation_metadata
  Nested message and enum types in BatchOperationMetadata.
• cluster_controller_client
  Generated client implementations.
• cluster_operation_status
  Nested message and enum types in ClusterOperationStatus.
• cluster_status
  Nested message and enum types in ClusterStatus.
• dataproc_metric_config
  Nested message and enum types in DataprocMetricConfig.
• execution_config
  Nested message and enum types in ExecutionConfig.
• gce_cluster_config
  Nested message and enum types in GceClusterConfig.
• gke_node_pool_config
  Nested message and enum types in GkeNodePoolConfig.
• gke_node_pool_target
  Nested message and enum types in GkeNodePoolTarget.
• hadoop_job
  Nested message and enum types in HadoopJob.
• hive_job
  Nested message and enum types in HiveJob.
• instance_flexibility_policy
  Nested message and enum types in InstanceFlexibilityPolicy.
• instance_group_config
  Nested message and enum types in InstanceGroupConfig.
• job
  Nested message and enum types in Job.
• job_controller_client
  Generated client implementations.
• job_status
  Nested message and enum types in JobStatus.
• jupyter_config
  Nested message and enum types in JupyterConfig.
• kubernetes_cluster_config
  Nested message and enum types in KubernetesClusterConfig.
• lifecycle_config
  Nested message and enum types in LifecycleConfig.
• list_jobs_request
  Nested message and enum types in ListJobsRequest.
• logging_config
  Nested message and enum types in LoggingConfig.
• node_group
  Nested message and enum types in NodeGroup.
• node_group_controller_client
  Generated client implementations.
• node_group_operation_metadata
  Nested message and enum types in NodeGroupOperationMetadata.
• ordered_job
  Nested message and enum types in OrderedJob.
• parameter_validation
  Nested message and enum types in ParameterValidation.
• pig_job
  Nested message and enum types in PigJob.
• presto_job
  Nested message and enum types in PrestoJob.
• reservation_affinity
  Nested message and enum types in ReservationAffinity.
• session
  Nested message and enum types in Session.
• session_controller_client
  Generated client implementations.
• session_operation_metadata
  Nested message and enum types in SessionOperationMetadata.
• session_template
  Nested message and enum types in SessionTemplate.
• session_template_controller_client
  Generated client implementations.
• spark_batch
  Nested message and enum types in SparkBatch.
• spark_job
  Nested message and enum types in SparkJob.
• spark_sql_job
  Nested message and enum types in SparkSqlJob.
• trino_job
  Nested message and enum types in TrinoJob.
• virtual_cluster_config
  Nested message and enum types in VirtualClusterConfig.
• workflow_metadata
  Nested message and enum types in WorkflowMetadata.
• workflow_node
  Nested message and enum types in WorkflowNode.
• workflow_template_placement
  Nested message and enum types in WorkflowTemplatePlacement.
• workflow_template_service_client
  Generated client implementations.
• yarn_application
  Nested message and enum types in YarnApplication.

Structs

• AcceleratorConfig
  Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine.
• AutoscalingConfig
  Autoscaling Policy config associated with the cluster.
• AutoscalingPolicy
  Describes an autoscaling policy for Dataproc cluster autoscaler.
• AuxiliaryNodeGroup
  Node group identification and configuration information.
• AuxiliaryServicesConfig
  Auxiliary services configuration for a Cluster.
• BasicAutoscalingAlgorithm
  Basic algorithm for autoscaling.
• BasicYarnAutoscalingConfig
  Basic autoscaling configurations for YARN.
• Batch
  A representation of a batch workload in the service.
• BatchOperationMetadata
  Metadata describing the Batch operation.
• CancelJobRequest
  A request to cancel a job.
• Cluster
  Describes the identifying information, config, and status of a Dataproc cluster
• ClusterConfig
  The cluster config.
• ClusterMetrics
  Contains cluster daemon metrics, such as HDFS and YARN stats.
• ClusterOperation
  The cluster operation triggered by a workflow.
• ClusterOperationMetadata
  Metadata describing the operation.
• ClusterOperationStatus
  The status of the operation.
• ClusterSelector
  A selector that chooses target cluster for jobs based on metadata.
• ClusterStatus
  The status of a cluster and its instances.
• ConfidentialInstanceConfig
  Confidential Instance Config for clusters using Confidential VMs
• CreateAutoscalingPolicyRequest
  A request to create an autoscaling policy.
• CreateBatchRequest
  A request to create a batch workload.
• CreateClusterRequest
  A request to create a cluster.
• CreateNodeGroupRequest
  A request to create a node group.
• CreateSessionRequest
  A request to create a session.
• CreateSessionTemplateRequest
  A request to create a session template.
• CreateWorkflowTemplateRequest
  A request to create a workflow template.
• DataprocMetricConfig
  Dataproc metric config.
• DeleteAutoscalingPolicyRequest
  A request to delete an autoscaling policy.
• DeleteBatchRequest
  A request to delete a batch workload.
• DeleteClusterRequest
  A request to delete a cluster.
• DeleteJobRequest
  A request to delete a job.
• DeleteSessionRequest
  A request to delete a session.
• DeleteSessionTemplateRequest
  A request to delete a session template.
• DeleteWorkflowTemplateRequest
  A request to delete a workflow template.
• DiagnoseClusterRequest
  A request to collect cluster diagnostic information.
• DiagnoseClusterResults
  The location of diagnostic output.
• DiskConfig
  Specifies the config of disk options for a group of VM instances.
• DriverSchedulingConfig
  Driver scheduling configuration.
• EncryptionConfig
  Encryption settings for the cluster.
• EndpointConfig
  Endpoint config for this cluster
• EnvironmentConfig
  Environment configuration for a workload.
• ExecutionConfig
  Execution configuration for a workload.
• GceClusterConfig
  Common config settings for resources of Compute Engine cluster instances, applicable to all instances in the cluster.
• GetAutoscalingPolicyRequest
  A request to fetch an autoscaling policy.
• GetBatchRequest
  A request to get the resource representation for a batch workload.
• GetClusterRequest
  Request to get the resource representation for a cluster in a project.
• GetJobRequest
  A request to get the resource representation for a job in a project.
• GetNodeGroupRequest
  A request to get a node group .
• GetSessionRequest
  A request to get the resource representation for a session.
• GetSessionTemplateRequest
  A request to get the resource representation for a session template.
• GetWorkflowTemplateRequest
  A request to fetch a workflow template.
• GkeClusterConfig
  The cluster’s GKE config.
• GkeNodePoolConfig
  The configuration of a GKE node pool used by a Dataproc-on-GKE cluster.
• GkeNodePoolTarget
  GKE node pools that Dataproc workloads run on.
• HadoopJob
  A Dataproc job for running Apache Hadoop MapReduce jobs on Apache Hadoop YARN.
• HiveJob
  A Dataproc job for running Apache Hive queries on YARN.
• IdentityConfig
  Identity related configuration, including service account based secure multi-tenancy user mappings.
• InstanceFlexibilityPolicy
  Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.
• InstanceGroupAutoscalingPolicyConfig
  Configuration for the size bounds of an instance group, including its proportional size to other groups.
• InstanceGroupConfig
  The config settings for Compute Engine resources in an instance group, such as a master or worker group.
• InstanceReference
  A reference to a Compute Engine instance.
• InstantiateInlineWorkflowTemplateRequest
  A request to instantiate an inline workflow template.
• InstantiateWorkflowTemplateRequest
  A request to instantiate a workflow template.
• Job
  A Dataproc job resource.
• JobMetadata
  Job Operation metadata.
• JobPlacement
  Dataproc job config.
• JobReference
  Encapsulates the full scoping used to reference a job.
• JobScheduling
  Job scheduling options.
• JobStatus
  Dataproc job status.
• JupyterConfig
  Jupyter configuration for an interactive session.
• KerberosConfig
  Specifies Kerberos related configuration.
• KubernetesClusterConfig
  The configuration for running the Dataproc cluster on Kubernetes.
• KubernetesSoftwareConfig
  The software configuration for this Dataproc cluster running on Kubernetes.
• LifecycleConfig
  Specifies the cluster auto-delete schedule configuration.
• ListAutoscalingPoliciesRequest
  A request to list autoscaling policies in a project.
• ListAutoscalingPoliciesResponse
  A response to a request to list autoscaling policies in a project.
• ListBatchesRequest
  A request to list batch workloads in a project.
• ListBatchesResponse
  A list of batch workloads.
• ListClustersRequest
  A request to list the clusters in a project.
• ListClustersResponse
  The list of all clusters in a project.
• ListJobsRequest
  A request to list jobs in a project.
• ListJobsResponse
  A list of jobs in a project.
• ListSessionTemplatesRequest
  A request to list session templates in a project.
• ListSessionTemplatesResponse
  A list of session templates.
• ListSessionsRequest
  A request to list sessions in a project.
• ListSessionsResponse
  A list of interactive sessions.
• ListWorkflowTemplatesRequest
  A request to list workflow templates in a project.
• ListWorkflowTemplatesResponse
  A response to a request to list workflow templates in a project.
• LoggingConfig
  The runtime logging config of the job.
• ManagedCluster
  Cluster that is managed by the workflow.
• ManagedGroupConfig
  Specifies the resources used to actively manage an instance group.
• MetastoreConfig
  Specifies a Metastore configuration.
• NodeGroup
  Dataproc Node Group. The Dataproc NodeGroup resource is not related to the Dataproc [NodeGroupAffinity][google.cloud.dataproc.v1.NodeGroupAffinity] resource.
• NodeGroupAffinity
  Node Group Affinity for clusters using sole-tenant node groups. The Dataproc NodeGroupAffinity resource is not related to the Dataproc [NodeGroup][google.cloud.dataproc.v1.NodeGroup] resource.
• NodeGroupOperationMetadata
  Metadata describing the node group operation.
• NodeInitializationAction
  Specifies an executable to run on a fully configured node and a timeout period for executable completion.
• OrderedJob
  A job executed by the workflow.
• ParameterValidation
  Configuration for parameter validation.
• PeripheralsConfig
  Auxiliary services configuration for a workload.
• PigJob
  A Dataproc job for running Apache Pig queries on YARN.
• PrestoJob
  A Dataproc job for running Presto queries. IMPORTANT: The Dataproc Presto Optional Component must be enabled when the cluster is created to submit a Presto job to the cluster.
• PyPiRepositoryConfig
  Configuration for PyPi repository
• PySparkBatch
  A configuration for running an Apache PySpark batch workload.
• PySparkJob
  A Dataproc job for running Apache PySpark applications on YARN.
• QueryList
  A list of queries to run on a cluster.
• RegexValidation
  Validation based on regular expressions.
• RepositoryConfig
  Configuration for dependency repositories
• ReservationAffinity
  Reservation Affinity for consuming Zonal reservation.
• ResizeNodeGroupRequest
  A request to resize a node group.
• RuntimeConfig
  Runtime configuration for a workload.
• RuntimeInfo
  Runtime information about workload execution.
• SecurityConfig
  Security related configuration, including encryption, Kerberos, etc.
• Session
  A representation of a session.
• SessionOperationMetadata
  Metadata describing the Session operation.
• SessionTemplate
  A representation of a session template.
• ShieldedInstanceConfig
  Shielded Instance Config for clusters using Compute Engine Shielded VMs.
• SoftwareConfig
  Specifies the selection and config of software inside the cluster.
• SparkBatch
  A configuration for running an Apache Spark batch workload.
• SparkHistoryServerConfig
  Spark History Server configuration for the workload.
• SparkJob
  A Dataproc job for running Apache Spark applications on YARN.
• SparkRBatch
  A configuration for running an Apache SparkR batch workload.
• SparkRJob
  A Dataproc job for running Apache SparkR applications on YARN.
• SparkSqlBatch
  A configuration for running Apache Spark SQL queries as a batch workload.
• SparkSqlJob
  A Dataproc job for running Apache Spark SQL queries.
• StartClusterRequest
  A request to start a cluster.
• StartupConfig
  Configuration to handle the startup of instances during cluster create and update process.
• StopClusterRequest
  A request to stop a cluster.
• SubmitJobRequest
  A request to submit a job.
• TemplateParameter
  A configurable parameter that replaces one or more fields in the template. Parameterizable fields:
• TerminateSessionRequest
  A request to terminate an interactive session.
• TrinoJob
  A Dataproc job for running Trino queries. IMPORTANT: The Dataproc Trino Optional Component must be enabled when the cluster is created to submit a Trino job to the cluster.
• UpdateAutoscalingPolicyRequest
  A request to update an autoscaling policy.
• UpdateClusterRequest
  A request to update a cluster.
• UpdateJobRequest
  A request to update a job.
• UpdateSessionTemplateRequest
  A request to update a session template.
• UpdateWorkflowTemplateRequest
  A request to update a workflow template.
• UsageMetrics
  Usage metrics represent approximate total resources consumed by a workload.
• UsageSnapshot
  The usage snapshot represents the resources consumed by a workload at a specified time.
• ValueValidation
  Validation based on a list of allowed values.
• VirtualClusterConfig
  The Dataproc cluster config for a cluster that does not directly control the underlying compute resources, such as a Dataproc-on-GKE cluster.
• WorkflowGraph
  The workflow graph.
• WorkflowMetadata
  A Dataproc workflow template resource.
• WorkflowNode
  The workflow node.
• WorkflowTemplate
  A Dataproc workflow template resource.
• WorkflowTemplatePlacement
  Specifies workflow execution target.
• YarnApplication
  A YARN application created by a job. Application information is a subset of org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto.

Enums

• Component
  Cluster components that can be activated.
• FailureAction
  Actions in response to failure of a resource associated with a cluster.