Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SchemaModelevaluationMetricsPairwiseTextGenerationEvaluationMetrics

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Metrics for general pairwise text generation evaluation results.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1SchemaModelevaluationMetricsPairwiseTextGenerationEvaluationMetrics

Returns a new instance of GoogleCloudAiplatformV1beta1SchemaModelevaluationMetricsPairwiseTextGenerationEvaluationMetrics.



25477
25478
25479
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25477

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#accuracy ⇒ Float

Fraction of cases where the autorater agreed with the human raters. Corresponds to the JSON property accuracy

Returns:

  • (Float) 


25405
25406
25407
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25405

def accuracy
  @accuracy
end

#baseline_model_win_rate ⇒ Float

Percentage of time the autorater decided the baseline model had the better response. Corresponds to the JSON property baselineModelWinRate

Returns:

  • (Float) 


25411
25412
25413
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25411

def baseline_model_win_rate
  @baseline_model_win_rate
end

#cohens_kappa ⇒ Float

A measurement of agreement between the autorater and human raters that takes the likelihood of random agreement into account. Corresponds to the JSON property cohensKappa

Returns:

  • (Float) 


25417
25418
25419
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25417

def cohens_kappa
  @cohens_kappa
end

#f1_score ⇒ Float

Harmonic mean of precision and recall. Corresponds to the JSON property f1Score

Returns:

  • (Float) 


25422
25423
25424
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25422

def f1_score
  @f1_score
end

#false_negative_count ⇒ Fixnum

Number of examples where the autorater chose the baseline model, but humans preferred the model. Corresponds to the JSON property falseNegativeCount

Returns:

  • (Fixnum) 


25428
25429
25430
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25428

def false_negative_count
  @false_negative_count
end

#false_positive_count ⇒ Fixnum

Number of examples where the autorater chose the model, but humans preferred the baseline model. Corresponds to the JSON property falsePositiveCount

Returns:

  • (Fixnum) 


25434
25435
25436
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25434

def false_positive_count
  @false_positive_count
end

#human_preference_baseline_model_win_rate ⇒ Float

Percentage of time humans decided the baseline model had the better response. Corresponds to the JSON property humanPreferenceBaselineModelWinRate

Returns:

  • (Float) 


25439
25440
25441
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25439

def human_preference_baseline_model_win_rate
  @human_preference_baseline_model_win_rate
end

#human_preference_model_win_rate ⇒ Float

Percentage of time humans decided the model had the better response. Corresponds to the JSON property humanPreferenceModelWinRate

Returns:

  • (Float) 


25444
25445
25446
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25444

def human_preference_model_win_rate
  @human_preference_model_win_rate
end

#model_win_rate ⇒ Float

Percentage of time the autorater decided the model had the better response. Corresponds to the JSON property modelWinRate

Returns:

  • (Float) 


25449
25450
25451
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25449

def model_win_rate
  @model_win_rate
end

#precision ⇒ Float

Fraction of cases where the autorater and humans thought the model had a better response out of all cases where the autorater thought the model had a better response. True positive divided by all positive. Corresponds to the JSON property precision

Returns:

  • (Float) 


25456
25457
25458
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25456

def precision
  @precision
end

#recall ⇒ Float

Fraction of cases where the autorater and humans thought the model had a better response out of all cases where the humans thought the model had a better response. Corresponds to the JSON property recall

Returns:

  • (Float) 


25463
25464
25465
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25463

def recall
  @recall
end

#true_negative_count ⇒ Fixnum

Number of examples where both the autorater and humans decided that the model had the worse response. Corresponds to the JSON property trueNegativeCount

Returns:

  • (Fixnum) 


25469
25470
25471
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25469

def true_negative_count
  @true_negative_count
end

#true_positive_count ⇒ Fixnum

Number of examples where both the autorater and humans decided that the model had the better response. Corresponds to the JSON property truePositiveCount

Returns:

  • (Fixnum) 


25475
25476
25477
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25475

def true_positive_count
  @true_positive_count
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



25482
25483
25484
25485
25486
25487
25488
25489
25490
25491
25492
25493
25494
25495
25496
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 25482

def update!(**args)
  @accuracy = args[:accuracy] if args.key?(:accuracy)
  @baseline_model_win_rate = args[:baseline_model_win_rate] if args.key?(:baseline_model_win_rate)
  @cohens_kappa = args[:cohens_kappa] if args.key?(:cohens_kappa)
  @f1_score = args[:f1_score] if args.key?(:f1_score)
  @false_negative_count = args[:false_negative_count] if args.key?(:false_negative_count)
  @false_positive_count = args[:false_positive_count] if args.key?(:false_positive_count)
  @human_preference_baseline_model_win_rate = args[:human_preference_baseline_model_win_rate] if args.key?(:human_preference_baseline_model_win_rate)
  @human_preference_model_win_rate = args[:human_preference_model_win_rate] if args.key?(:human_preference_model_win_rate)
  @model_win_rate = args[:model_win_rate] if args.key?(:model_win_rate)
  @precision = args[:precision] if args.key?(:precision)
  @recall = args[:recall] if args.key?(:recall)
  @true_negative_count = args[:true_negative_count] if args.key?(:true_negative_count)
  @true_positive_count = args[:true_positive_count] if args.key?(:true_positive_count)
end