Class EvaluationJobConfig

Configures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.

Inheritance

System.Object

EvaluationJobConfig

Implements

IMessage<EvaluationJobConfig>

System.IEquatable<EvaluationJobConfig>

IDeepCloneable<EvaluationJobConfig>

Google.Protobuf.IBufferMessage

IMessage

Inherited Members

System.Object.ToString()

System.Object.GetHashCode()

System.Object.GetType()

System.Object.MemberwiseClone()

Namespace: Google.Cloud.DataLabeling.V1Beta1

Assembly: Google.Cloud.DataLabeling.V1Beta1.dll

Syntax

public sealed class EvaluationJobConfig : IMessage<EvaluationJobConfig>, IEquatable<EvaluationJobConfig>, IDeepCloneable<EvaluationJobConfig>, IBufferMessage, IMessage

Constructors

EvaluationJobConfig()

Declaration

public EvaluationJobConfig()

EvaluationJobConfig(EvaluationJobConfig)

Declaration

public EvaluationJobConfig(EvaluationJobConfig other)

Parameters

Type	Name	Description
EvaluationJobConfig	other

Properties

BigqueryImportKeys

Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

You can provide the following entries in this field:

• data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
• reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
• label_json_key: the label key for prediction output. Required.
• label_score_json_key: the score key for prediction output. Required.
• bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection.

Learn how to configure prediction keys.

Declaration

public MapField<string, string> BigqueryImportKeys { get; }

Property Value

Type	Description
MapField<System.String, System.String>

BoundingPolyConfig

Specify this field if your model version performs image object detection (bounding box detection).

annotationSpecSet in this configuration must match [EvaluationJob.annotationSpecSet][google.cloud.datalabeling.v1beta1.EvaluationJob.annotation_spec_set].

Declaration

public BoundingPolyConfig BoundingPolyConfig { get; set; }

Property Value

Type	Description
BoundingPolyConfig

EvaluationConfig

Required. Details for calculating evaluation metrics and creating [Evaulations][google.cloud.datalabeling.v1beta1.Evaluation]. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Declaration

public EvaluationConfig EvaluationConfig { get; set; }

Property Value

Type	Description
EvaluationConfig

EvaluationJobAlertConfig

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Declaration

public EvaluationJobAlertConfig EvaluationJobAlertConfig { get; set; }

Property Value

Type	Description
EvaluationJobAlertConfig

ExampleCount

Required. The maximum number of predictions to sample and save to BigQuery during each [evaluation interval][google.cloud.datalabeling.v1beta1.EvaluationJob.schedule]. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Declaration

public int ExampleCount { get; set; }

Property Value

Type	Description
System.Int32

ExampleSamplePercentage

Required. Fraction of predictions to sample and save to BigQuery during each [evaluation interval][google.cloud.datalabeling.v1beta1.EvaluationJob.schedule]. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Declaration

public double ExampleSamplePercentage { get; set; }

Property Value

Type	Description
System.Double

HumanAnnotationConfig

Optional. Details for human annotation of your data. If you set [labelMissingGroundTruth][google.cloud.datalabeling.v1beta1.EvaluationJob.label_missing_ground_truth] to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

Note that you must create an [Instruction][google.cloud.datalabeling.v1beta1.Instruction] resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Declaration

public HumanAnnotationConfig HumanAnnotationConfig { get; set; }

Property Value

Type	Description
HumanAnnotationConfig

HumanAnnotationRequestConfigCase

Declaration

public EvaluationJobConfig.HumanAnnotationRequestConfigOneofCase HumanAnnotationRequestConfigCase { get; }

Property Value

Type	Description
EvaluationJobConfig.HumanAnnotationRequestConfigOneofCase

ImageClassificationConfig

Specify this field if your model version performs image classification or general classification.

annotationSpecSet in this configuration must match [EvaluationJob.annotationSpecSet][google.cloud.datalabeling.v1beta1.EvaluationJob.annotation_spec_set]. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in [input_config][google.cloud.datalabeling.v1beta1.EvaluationJobConfig.input_config].

Declaration

public ImageClassificationConfig ImageClassificationConfig { get; set; }

Property Value

Type	Description
ImageClassificationConfig

InputConfig

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

• dataType must be one of IMAGE, TEXT, or GENERAL_DATA.
• annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).
• If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
• You must specify bigquerySource (not gcsSource).

Declaration

public InputConfig InputConfig { get; set; }

Property Value

Type	Description
InputConfig

TextClassificationConfig

Specify this field if your model version performs text classification.

annotationSpecSet in this configuration must match [EvaluationJob.annotationSpecSet][google.cloud.datalabeling.v1beta1.EvaluationJob.annotation_spec_set]. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in [input_config][google.cloud.datalabeling.v1beta1.EvaluationJobConfig.input_config].

Declaration

public TextClassificationConfig TextClassificationConfig { get; set; }

Property Value

Type	Description
TextClassificationConfig