Struct googapis::google::cloud::aiplatform::v1::InputDataConfig [−][src]
pub struct InputDataConfig {
pub dataset_id: String,
pub annotations_filter: String,
pub annotation_schema_uri: String,
pub split: Option<Split>,
pub destination: Option<Destination>,
}
Expand description
Specifies Vertex AI owned input data to be used for training, and possibly evaluating, the Model.
Fields
dataset_id: String
Required. The ID of the Dataset in the same Project and Location which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.
annotations_filter: String
Applicable only to Datasets that have DataItems and Annotations.
A filter on Annotations of the Dataset. Only Annotations that both match this filter and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on (for the auto-assigned that role is decided by Vertex AI). A filter with same syntax as the one used in [ListAnnotations][google.cloud.aiplatform.v1.DatasetService.ListAnnotations] may be used, but note here it filters across all Annotations of the Dataset, and not just within a single DataItem.
annotation_schema_uri: String
Applicable only to custom training with Datasets that have DataItems and Annotations.
Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/ , note that the chosen schema must be consistent with [metadata][google.cloud.aiplatform.v1.Dataset.metadata_schema_uri] of the Dataset specified by [dataset_id][google.cloud.aiplatform.v1.InputDataConfig.dataset_id].
Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.
When used in conjunction with [annotations_filter][google.cloud.aiplatform.v1.InputDataConfig.annotations_filter], the Annotations used for training are filtered by both [annotations_filter][google.cloud.aiplatform.v1.InputDataConfig.annotations_filter] and [annotation_schema_uri][google.cloud.aiplatform.v1.InputDataConfig.annotation_schema_uri].
split: Option<Split>
The instructions how the input data should be split between the training, validation and test sets. If no split type is provided, the [fraction_split][google.cloud.aiplatform.v1.InputDataConfig.fraction_split] is used by default.
destination: Option<Destination>
Only applicable to Custom and Hyperparameter Tuning TrainingPipelines.
The destination of the training data to be written to.
Supported destination file formats:
- For non-tabular data: “jsonl”.
- For tabular data: “csv” and “bigquery”.
The following Vertex AI environment variables are passed to containers or python modules of the training task when this field is set:
- AIP_DATA_FORMAT : Exported data format.
- AIP_TRAINING_DATA_URI : Sharded exported training data uris.
- AIP_VALIDATION_DATA_URI : Sharded exported validation data uris.
- AIP_TEST_DATA_URI : Sharded exported test data uris.
Trait Implementations
fn merge_field<B>(
&mut self,
tag: u32,
wire_type: WireType,
buf: &mut B,
ctx: DecodeContext
) -> Result<(), DecodeError> where
B: Buf,
Returns the encoded length of the message without a length delimiter.
Encodes the message to a buffer. Read more
Encodes the message to a newly allocated buffer.
Encodes the message with a length-delimiter to a buffer. Read more
Encodes the message with a length-delimiter to a newly allocated buffer.
Decodes an instance of the message from a buffer. Read more
fn decode_length_delimited<B>(buf: B) -> Result<Self, DecodeError> where
Self: Default,
B: Buf,
fn decode_length_delimited<B>(buf: B) -> Result<Self, DecodeError> where
Self: Default,
B: Buf,
Decodes a length-delimited instance of the message from the buffer.
Decodes an instance of the message from a buffer, and merges it into self
. Read more
Decodes a length-delimited instance of the message from buffer, and
merges it into self
. Read more
This method tests for self
and other
values to be equal, and is used
by ==
. Read more
This method tests for !=
.
Auto Trait Implementations
impl RefUnwindSafe for InputDataConfig
impl Send for InputDataConfig
impl Sync for InputDataConfig
impl Unpin for InputDataConfig
impl UnwindSafe for InputDataConfig
Blanket Implementations
Mutably borrows from an owned value. Read more
Wrap the input message T
in a tonic::Request
pub fn vzip(self) -> V
Attaches the provided Subscriber
to this type, returning a
WithDispatch
wrapper. Read more
Attaches the current default Subscriber
to this type, returning a
WithDispatch
wrapper. Read more