OCDocker.OCScore.DNN.future.metrics module

Metrics utilities for the future DNN pipeline.

OCDocker.OCScore.DNN.future.metrics.compute_classification_metrics(y_true, y_score, target_ids=None, k_fractions=(0.01, 0.05, 0.1, 0.25, 0.5, 0.75))[source]

Compute classification and ranking metrics for a labeled ranking dataset.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

  • target_ids (np.ndarray | None, optional) – Target identifiers per sample, by default None.

  • k_fractions (Sequence[float], optional) – Fractions for top-k metrics, by default (0.01, 0.05, 0.10, 0.25, 0.50, 0.75).

Returns:

Metrics dictionary.

Return type:

Dict[str, float]

OCDocker.OCScore.DNN.future.metrics.compute_group_metrics(y_true, y_score, target_ids, k_fractions=(0.01, 0.05, 0.1, 0.25, 0.5, 0.75))[source]

Compute ranking metrics per target and macro-average them.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

  • target_ids (np.ndarray) – Target identifiers per sample.

  • k_fractions (Sequence[float], optional) – Fractions for top-k metrics, by default (0.01, 0.05, 0.10, 0.25, 0.50, 0.75).

Returns:

Macro-averaged ranking metrics.

Return type:

Dict[str, float]

OCDocker.OCScore.DNN.future.metrics.ndcg_at_k(y_true, y_score, k)[source]

Compute NDCG@k.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

  • k (int) – Rank cutoff.

Returns:

NDCG@k value.

Return type:

float

OCDocker.OCScore.DNN.future.metrics.partial_auc(y_true, y_score, max_fpr=0.05)[source]

Compute partial AUC up to max_fpr.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

  • max_fpr (float, optional) – Maximum false positive rate, by default 0.05.

Returns:

Normalized partial AUC.

Return type:

float

OCDocker.OCScore.DNN.future.metrics.safe_auc(y_true, y_score)[source]

Compute ROC AUC with guards for degenerate labels.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

Returns:

ROC AUC value or 0.0 if undefined.

Return type:

float

OCDocker.OCScore.DNN.future.metrics.safe_log_loss(y_true, y_score)[source]

Compute log-loss with guards for degenerate labels.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

Returns:

Log-loss value or inf if undefined.

Return type:

float

OCDocker.OCScore.DNN.future.metrics.safe_pr_auc(y_true, y_score)[source]

Compute PR AUC with guards for degenerate labels.

Parameters:

  • y_true (np.ndarray) – Ground-truth labels.

  • y_score (np.ndarray) – Predicted scores.

Returns:

PR AUC value or 0.0 if undefined.

Return type:

float