OCDocker.OCScore.Analysis.ProductionBaselines module

Production-grade baseline evaluation on DUDEz screening splits.

Evaluates individual scoring functions and train-only sklearn learners on the same row indices and ranking metrics used by the staged OCScore protocol.

OCDocker.OCScore.Analysis.ProductionBaselines.PRODUCTION_BASELINE_RANK_METRICS = ('BEDROC', 'ROC-AUC', 'PR-AUC', 'EF1%', 'EF5%', 'NDCG@1%', 'NDCG@5%')

Copyright (c) Federal University of Rio de Janeiro (UFRJ), Artur Duque Rossi, and Pedro Henrique Monteiro Torres.

SPDX-License-Identifier: BSD-3-Clause

See the LICENSE file for full terms.

class OCDocker.OCScore.Analysis.ProductionBaselines.ProductionBaselineConfig(label_column='label', group_column='receptor', random_seed=42, include_xgb=True, include_lgbm=True, include_shuffle_control=True, include_sf_consensus=True, include_descriptor_aggregates=True, metric_names=<factory>, bedroc_alpha=20.0)[source]

Bases: object

Configuration for production-grade baseline evaluation.

Parameters:

  • label_column (str, optional) – Binary label column in the DUDEz dataframe, by default "label".

  • group_column (str, optional) – Receptor/group column for grouped screening metrics, by default "receptor".

  • random_seed (int, optional) – Random seed for sklearn learners and shuffle control, by default 42.

  • include_xgb (bool, optional) – Attempt XGBoost baseline when importable, by default True.

  • include_lgbm (bool, optional) – Attempt LightGBM baseline when importable, by default True.

  • include_shuffle_control (bool, optional) – Include shuffled-label logistic regression control, by default True.

  • metric_names (Sequence[str], optional) – Ranking metrics retained in outputs, by default production headline metrics.

  • bedroc_alpha (float, optional) – BEDROC exponential weighting factor, by default 20.0.

  • include_sf_consensus (bool)

  • include_descriptor_aggregates (bool)

label_column: str = 'label'
group_column: str = 'receptor'
random_seed: int = 42
include_xgb: bool = True
include_lgbm: bool = True
include_shuffle_control: bool = True
include_sf_consensus: bool = True
include_descriptor_aggregates: bool = True
metric_names: Sequence[str]
bedroc_alpha: float = 20.0
exception OCDocker.OCScore.Analysis.ProductionBaselines.TrainOnlyFitError[source]

Bases: ValueError

Raised when a baseline fit uses non-train row indices.

OCDocker.OCScore.Analysis.ProductionBaselines.aggregate_baseline_rows(rows)[source]

Aggregate per-replica baseline rows to median metrics per baseline and split.

Parameters:

rows (Sequence[Mapping[str, Any]])

Return type:

DataFrame

OCDocker.OCScore.Analysis.ProductionBaselines.build_baseline_rank_table(summary_df, *, split='test', metric='BEDROC')[source]

Rank baselines by aggregated test-split metric (higher is better).

Parameters:

  • summary_df (DataFrame)

  • split (str)

  • metric (str)

Return type:

DataFrame

OCDocker.OCScore.Analysis.ProductionBaselines.evaluate_individual_sf_baselines(dataframe, selected_features, split_indices, *, label_column, group_column, config)[source]

Evaluate raw scoring-function columns on validation and test splits.

Parameters:

  • dataframe (DataFrame)

  • selected_features (Sequence[str])

  • split_indices (Mapping[str, Sequence[int]])

  • label_column (str)

  • group_column (str)

  • config (ProductionBaselineConfig)

Return type:

list[dict[str, Any]]

OCDocker.OCScore.Analysis.ProductionBaselines.evaluate_sf_consensus_baselines(dataframe, selected_features, split_indices, *, label_column, group_column, config)[source]

Evaluate SF consensus row aggregates on validation and test splits.

Parameters:

  • dataframe (DataFrame)

  • selected_features (Sequence[str])

  • split_indices (Mapping[str, Sequence[int]])

  • label_column (str)

  • group_column (str)

  • config (ProductionBaselineConfig)

Return type:

list[dict[str, Any]]

OCDocker.OCScore.Analysis.ProductionBaselines.evaluate_descriptor_aggregate_baselines(dataframe, selected_features, split_indices, *, label_column, group_column, config)[source]

Evaluate descriptor row aggregates on validation and test splits.

Parameters:

  • dataframe (DataFrame)

  • selected_features (Sequence[str])

  • split_indices (Mapping[str, Sequence[int]])

  • label_column (str)

  • group_column (str)

  • config (ProductionBaselineConfig)

Return type:

list[dict[str, Any]]

OCDocker.OCScore.Analysis.ProductionBaselines.evaluate_learned_sf_baselines(dataframe, selected_features, split_indices, *, label_column, group_column, config, fit_indices=None, shuffle_train_labels=False)[source]

Evaluate train-only sklearn baselines on SF descriptor columns.

Returns:

Per-split metric rows and skip notes for optional learners.

Return type:

tuple[list[dict[str, Any]], list[dict[str, str]]]

Parameters:

  • dataframe (DataFrame)

  • selected_features (Sequence[str])

  • split_indices (Mapping[str, Sequence[int]])

  • label_column (str)

  • group_column (str)

  • config (ProductionBaselineConfig)

  • fit_indices (ndarray | None)

  • shuffle_train_labels (bool)

OCDocker.OCScore.Analysis.ProductionBaselines.run_and_write_production_baselines(output_dir, *, dudez_df, selected_features, replica_results, config=None)[source]

Evaluate baselines for all successful replicas and write CSV outputs.

Parameters:
Return type:

dict[str, str]

OCDocker.OCScore.Analysis.ProductionBaselines.run_production_baselines_for_replica(*, replica_name, dudez_df, selected_features, split_indices, config=None)[source]

Run all configured baselines for one successful replica.

Parameters:

  • replica_name (str)

  • dudez_df (DataFrame)

  • selected_features (Sequence[str])

  • split_indices (Mapping[str, Sequence[int]])

  • config (ProductionBaselineConfig | None)

Return type:

dict[str, Any]

OCDocker.OCScore.Analysis.ProductionBaselines.validate_fit_uses_train_only(train_indices, fit_indices)[source]

Ensure baseline fitting uses only train split row indices.

Parameters:

  • train_indices (np.ndarray) – Row indices in the train split.

  • fit_indices (np.ndarray) – Row indices used for fitting the baseline.

Raises:

TrainOnlyFitError – When fit_indices contains rows outside train_indices.

Return type:

None

OCDocker.OCScore.Analysis.ProductionBaselines.write_production_baseline_reports(output_dir, per_replica_results, *, skip_notes=None)[source]

Write baseline CSV reports under output_dir.

Parameters:

  • output_dir (str | Path)

  • per_replica_results (Sequence[Mapping[str, Any]])

  • skip_notes (Sequence[Mapping[str, str]] | None)

Return type:

dict[str, str]