OCDocker.OCScore.Analysis.Plotting.Stats module

Plotting helpers for statistical summaries (scatter/box/bar, diagnostics, PCA importance). These utilities are used by Analysis workflows and StatTests.

Usage:

import OCDocker.OCScore.Analysis.Plotting.Stats as ocstatplot

OCDocker.OCScore.Analysis.Plotting.Stats.plot_ablation_bedroc_significance_bars(significance_df, *, reference_policy='full_ocscore', metric_label='BEDROC', output_dir='plots', alpha=0.05)[source]

Plot per-policy BEDROC means vs a reference policy, colored by paired significance.

Parameters:

  • significance_df (pd.DataFrame) – Output of OCScore.Analysis.AblationSignificance.compute_ablation_significance (expects columns ‘policy’, ‘reference_mean’, ‘policy_mean’, ‘mean_diff’, ‘pvalue_corrected’, ‘reject_null’).

  • reference_policy (str) – Name of the reference policy, used for the axis label and reference line. Default: ‘full_ocscore’.

  • metric_label (str) – Metric label for titling. Default: ‘BEDROC’.

  • output_dir (str) – Where to save the plot image. Default: ‘plots’.

  • alpha (float) – Family-wise significance threshold used only for the subtitle text. Default: 0.05.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.classify_policies_by_shortcut_rule(plot_df, *, reference_policy='full_ocscore', risk_threshold=20.0, bedroc_column='bedroc_mean', risk_column='shortcut_risk_max_pct')[source]

Split policies into retained and discarded by the shortcut-risk rule.

A policy is discarded when it beats the reference policy’s mean metric and concentrates risk_threshold percent or more of its total SHAP importance in a single feature: the gain is real but rides on one dominant feature. Policies that do not beat the reference are not candidates, so the rule does not apply to them and they belong to neither group.

Parameters:

  • plot_df (pd.DataFrame) – One row per policy, with a 'policy' column plus bedroc_column and risk_column.

  • reference_policy (str) – Policy whose mean metric defines the candidacy cutoff. Default: ‘full_ocscore’.

  • risk_threshold (float) – Maximum single-feature SHAP share, in percent, tolerated in a candidate. Default: 20.0.

  • bedroc_column (str) – Column holding the per-policy mean metric. Default: ‘bedroc_mean’.

  • risk_column (str) – Column holding the per-policy shortcut risk, in percent. Default: ‘shortcut_risk_max_pct’.

Returns:

Retained (low-risk) and discarded (high-risk) candidate policy names.

Return type:

tuple[list[str], list[str]]

Raises:

ValueError – If reference_policy is absent from plot_df.

OCDocker.OCScore.Analysis.Plotting.Stats.classify_policies_by_eligibility_and_shortcut_risk(plot_df, *, reference_policy='full_ocscore', eligibility_column='eligible', risk_threshold=20.0, risk_column='shortcut_risk_max_pct')[source]

Split statistically eligible policies by their shortcut risk.

Unlike classify_policies_by_shortcut_rule(), eligibility is supplied explicitly instead of being inferred from whether the plotted mean exceeds the reference mean. This is appropriate when candidacy comes from a paired significance test while the scatter axis shows the validation-set metric.

Parameters:

  • plot_df (pandas.DataFrame) – One row per policy, including 'policy', eligibility_column and risk_column.

  • reference_policy (str) – Reference policy, excluded from both returned groups. Default: ‘full_ocscore’.

  • eligibility_column (str) – Boolean column identifying policies that passed the formal performance screen. Default: ‘eligible’.

  • risk_threshold (float) – Maximum single-feature SHAP share tolerated in a retained policy, in percent. Default: 20.0.

  • risk_column (str) – Column holding shortcut risk, in percent. Default: ‘shortcut_risk_max_pct’.

Returns:

Retained (eligible and low-risk) and discarded (eligible and high-risk) policy names.

Return type:

tuple[list[str], list[str]]

Raises:

ValueError – If the eligibility or risk column is absent.

OCDocker.OCScore.Analysis.Plotting.Stats.plot_bedroc_vs_shortcut_risk_scatter(plot_df, *, reference_policy='full_ocscore', good_policies=None, bad_policies=None, show_rule_geometry=None, risk_threshold=20.0, bedroc_column='bedroc_mean', risk_column='shortcut_risk_max_pct', label_column=None, highlight_policy=None, metric_label='BEDROC', title=None, xlabel=None, ylabel='Shortcut risk\n(max. % of SHAP importance in a single feature)', legend_labels=None, threshold_note=None, zone_note=None, reference_note=None, highlight_note=None, label_offsets=None, break_x_axis=True, figsize=(6.9, 4.6), dpi=300, output_dir='plots')[source]

Scatter per-policy mean BEDROC against SHAP shortcut risk, under the shortcut rule.

By default, point colors are derived from whether a policy beats the reference mean and from risk_threshold. Explicit good_policies / bad_policies may instead provide groups obtained from an independent eligibility rule, such as a Holm-corrected paired validation test. In that case the mean-reference geometry is hidden by default because it is contextual, not a decision cutoff.

When one policy sits far from every other on the x axis (a low-signal control, typically), it compresses the interesting cluster into a fraction of the width. break_x_axis splits the axis across that empty region instead, keeping every point visible.

Parameters:

  • plot_df (pd.DataFrame) – One row per policy, with a 'policy' column plus bedroc_column and risk_column (e.g. the output of SHAP.Dominance.aggregate_dominant_feature_risk merged with per-policy mean BEDROC).

  • reference_policy (str) – Policy plotted as the reference marker, with a dotted vertical guide at its metric value. Default: ‘full_ocscore’.

  • good_policies (sequence[str] | None, optional) – Overrides the retained group. Derived from the rule when None. Default: None.

  • bad_policies (sequence[str] | None, optional) – Overrides the discarded group. Derived from the rule when None. Default: None.

  • show_rule_geometry (bool | None, optional) – Draw the “beats the reference” vertical guide and the shaded discard quadrant, both of which visualize the derived rule’s x-axis condition. When good_policies/bad_policies override that rule (e.g. coloring by a statistical eligibility test while the x axis plots the corresponding mean), a point can legitimately sit on the “wrong” side of that geometry, which reads as a contradiction. Defaults to True when the grouping is derived (no override) and False when either override is supplied; pass explicitly to force either behavior. The horizontal risk-threshold guide is unaffected, since it always matches risk_threshold regardless of grouping source. Default: None.

  • risk_threshold (float) – Shortcut-risk cutoff, in percent; also the horizontal guide line. Default: 20.0.

  • bedroc_column (str) – Column with the per-policy mean metric value. Default: ‘bedroc_mean’.

  • risk_column (str) – Column with the per-policy shortcut-risk value. Default: ‘shortcut_risk_max_pct’.

  • label_column (str | None, optional) – Column used to annotate each point (falls back to 'policy'). Default: None.

  • highlight_policy (str | None, optional) – Policy to call out with an arrow and a bold label (the final recommendation, typically). Default: None.

  • metric_label (str) – Metric name, used in the default x label and in the output filename. Default: ‘BEDROC’.

  • title (str | None, optional) – Plot text. Override to render the figure in another language.

  • xlabel (str | None, optional) – Plot text. Override to render the figure in another language.

  • ylabel (str | None, optional) – Plot text. Override to render the figure in another language.

  • legend_labels (mapping[str, str] | None, optional) – Legend text, keyed by 'reference', 'retained', 'discarded' and 'other'. Override to render the figure in another language.

  • threshold_note (str | None, optional) – In-plot annotations for the risk guide line, the discard quadrant, the reference guide line and the highlighted policy. Override to render the figure in another language.

  • zone_note (str | None, optional) – In-plot annotations for the risk guide line, the discard quadrant, the reference guide line and the highlighted policy. Override to render the figure in another language.

  • reference_note (str | None, optional) – In-plot annotations for the risk guide line, the discard quadrant, the reference guide line and the highlighted policy. Override to render the figure in another language.

  • highlight_note (str | None, optional) – In-plot annotations for the risk guide line, the discard quadrant, the reference guide line and the highlighted policy. Override to render the figure in another language.

  • label_offsets (mapping[str, tuple[float, float]] | None, optional) – Per-policy (dx, dy) label offset override, in points, for policies whose default offset collides with a nearby marker or label. Overridden labels receive a subtle leader line back to their marker. Default: None.

  • break_x_axis (bool, optional) – Split the x axis across a wide empty region when one is present. Default: True.

  • figsize (tuple[float, float]) – Figure size in inches, sized to be embedded at roughly 1:1. Default: (6.9, 4.6).

  • dpi (int) – Figure DPI. Default: 300.

  • output_dir (str) – Where to save the plot image. Default: ‘plots’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_bar_with_significance(gh_df, metric, y_col='diff', colour_mapping=None, output_dir='plots', top_n=30)[source]

Plot Games-Howell pairwise differences as a horizontal bar chart.

Parameters:

  • gh_df (pd.DataFrame) – Output of pingouin.pairwise_gameshowell (expects columns ‘A’,’B’,’diff’,’pval’).

  • metric (str) – Metric label for titling (‘AUC’ or ‘RMSE’).

  • y_col (str) – Which column from gh_df to plot as bar length (default ‘diff’).

  • colour_mapping (dict | None, optional) – Unused here, accepted for API compatibility. Default: None.

  • output_dir (str) – Where to save the plot image. Default: ‘plots’.

  • top_n (int | None, optional) – If given, keep the top-N pairs by smallest p-value. Default: 30.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_barplots(df, n_trials, colour_mapping, output_dir)[source]

Generate sorted barplots of mean RMSE and AUC across methodologies with annotations.

Parameters:

  • df (pd.DataFrame) – Data containing ‘RMSE’, ‘AUC’, and ‘Methodology’.

  • n_trials (int) – Trial number for title and output naming.

  • colour_mapping (dict[str, tuple[float, float, float]]) – Dictionary mapping methodologies to colors.

  • output_dir (str) – Directory to save the barplot images.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_boxplots(df, n_trials, colour_mapping, output_dir, show_simple_consensus=False)[source]

Generate enhanced boxplots of RMSE and AUC across methodologies, with group shading and mean lines.

Parameters:

  • df (pd.DataFrame) – Data containing ‘RMSE’, ‘AUC’, and ‘Methodology’.

  • n_trials (int) – Number of trials used for title and filenames.

  • colour_mapping (dict[str, tuple[float, float, float]]) – Dictionary mapping methodologies to colors.

  • output_dir (str) – Directory to save the boxplot images.

  • show_simple_consensus (bool) – Whether to include consensus methodologies (any label ending with “consensus”).

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_combined_metric_scatter(df, n_trials, colour_mapping, output_dir, alpha=0.9)[source]

Generate a detailed scatter plot showing RMSE vs AUC across methods with shading and symbol cues.

Parameters:

  • df (pd.DataFrame) – DataFrame with RMSE, AUC, and Methodology columns.

  • n_trials (int) – Number of top trials considered.

  • colour_mapping (dict[str, tuple[float, float, float]]) – Dictionary mapping methodologies to colors.

  • output_dir (str) – Directory to save the scatter plot image.

  • alpha (float, optional) – Transparency for the markers. Default is 0.9.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_heatmap(gh_df, title, metric, output_dir='plots')[source]

Heatmap of Games-Howell p-values across methodology pairs.

Parameters:

  • gh_df (pd.DataFrame) – Output of pingouin.pairwise_gameshowell (expects columns ‘A’,’B ‘diff’,’pval’).

  • title (str) – Title for the heatmap.

  • metric (str) – Metric label for titling (‘AUC’ or ‘RMSE’).

  • output_dir (str) – Where to save the plot image. Default: ‘plots’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_normality_and_variance_diagnostics(df, metric, n_trials, output_dir='plots')[source]

Perform and plot normality and variance diagnostics across methodologies.

Quick diagnostics across groups: - Shapiro-Wilk p-values per methodology (bar of -log10 p) - Group variances (bar) and Levene’s p-value annotated

Parameters:

  • df (pd.DataFrame) – Data containing ‘Methodology’ and the specified metric.

  • metric (str) – Metric column to analyze (e.g., ‘AUC’ or ‘RMSE’).

  • n_trials (int) – Number of trials for title and output naming.

  • output_dir (str) – Directory to save the diagnostics plot. Default: ‘plots’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_pca_importance_barplot(importance_df, pca_type, n_features, n_trials, output_dir='plots')[source]

Barplot of top-N PCA feature importances.

Parameters:

  • importance_df (pd.DataFrame) – DataFrame with ‘Feature’ and ‘Importance’ columns.

  • pca_type (str) – PCA type label for titling (e.g., ‘1’, ‘2’).

  • n_features (int) – Number of top features to display.

  • n_trials (int) – Number of trials for title and output naming.

  • output_dir (str) – Directory to save the barplot image. Default: ‘plots’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_pca_importance_histogram(importance_df, pca_type, n_trials, output_dir='plots')[source]

Histogram of PCA feature importances.

Parameters:

  • importance_df (pd.DataFrame) – DataFrame with ‘Feature’ and ‘Importance’ columns.

  • pca_type (str) – PCA type label for titling (e.g., ‘1’, ‘2’).

  • n_trials (int) – Number of trials for title and output naming.

  • output_dir (str) – Directory to save the histogram image. Default: ‘plots’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.plot_scatterplot(df_rmse, df_auc, df_all, n_trials, colour_mapping, output_dir, orientation='horizontal', alpha=0.9)[source]

Create scatter plots of RMSE vs AUC for all methods and filtered subsets.

Create a 1x3 panel of scatter plots (RMSE vs AUC): - All filtered points - RMSE-filtered subset - AUC-filtered subset

Parameters:

  • df_all (pd.DataFrame) – DataFrame with all filtered points.

  • df_rmse (pd.DataFrame) – DataFrame filtered by RMSE threshold.

  • df_auc (pd.DataFrame) – DataFrame filtered by AUC threshold.

  • n_trials (int) – Number of top trials considered.

  • colour_mapping (dict[str, tuple[float, float, float]]) – Dictionary mapping methodologies to colors.

  • output_dir (str) – Directory to save the scatter plot image.

  • orientation (str, optional) – Orientation of the scatter plot. Default is ‘horizontal’. Options: ‘horizontal’, ‘vertical’.

  • alpha (float, optional) – Transparency for the markers. Default is 0.9.

Raises:

ValueError – If the orientation parameter is not ‘horizontal’ or ‘vertical’.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.save_pca_importance_bins(importance_df, pca_type, n_trials, output_dir='plots', n_bins=10)[source]

Assign quantile bins (qcut) and save as CSV.

Parameters:

  • importance_df (pd.DataFrame) – DataFrame with ‘Feature’ and ‘Importance’ columns.

  • pca_type (str) – PCA type label for titling (e.g., ‘1’, ‘2’).

  • n_trials (int) – Number of trials for title and output naming.

  • output_dir (str) – Directory to save the plot image. Default: ‘plots’.

  • n_bins (int) – Number of quantile bins to create. Default: 10.

Return type:

None

OCDocker.OCScore.Analysis.Plotting.Stats.save_pca_importance_groups(importance_df, pca_type, n_trials, output_dir='plots')[source]

Assign coarse groups by quantiles and save as CSV.

Parameters:

  • importance_df (pd.DataFrame) – DataFrame with ‘Feature’ and ‘Importance’ columns.

  • pca_type (str) – PCA type label for titling (e.g., ‘1’, ‘2’).

  • n_trials (int) – Number of trials for title and output naming.

  • output_dir (str) – Directory to save the plot image. Default: ‘plots’.

Return type:

None