Class GoogleCloudAiplatformV1CustomCodeExecutionSpec

Specificies a metric that is populated by evaluating user-defined Python code.

Inheritance

object

GoogleCloudAiplatformV1CustomCodeExecutionSpec

Implements

IDirectResponseSchema

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Google.Apis.Aiplatform.v1.Data

Assembly: Google.Apis.Aiplatform.v1.dll

Syntax

public class GoogleCloudAiplatformV1CustomCodeExecutionSpec : IDirectResponseSchema

Properties

ETag

The ETag of the item.

Declaration

public virtual string ETag { get; set; }

Property Value

Type	Description
string

EvaluationFunction

Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -> float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input:

instance= EvaluationInstance(
response=EvaluationInstance.InstanceData(text="The answer is 4."),
reference=EvaluationInstance.InstanceData(text="4") )

Example converted input:

{ 'response':
{'text': 'The answer is 4.'}, 'reference': {'text': '4'} }

Example python function:

def
evaluate(instance: dict[str, Any]) -> float: if instance'response' == instance'reference': return 1.0
return 0.0

CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.

Declaration

[JsonProperty("evaluationFunction")]
public virtual string EvaluationFunction { get; set; }

Property Value

Type	Description
string

Implements

IDirectResponseSchema