API reference — Metrics¶
Complete evaluation reports at once. Guides: Metrics, Evaluation reports.
Metrics¶
pythonclass Metrics(())
One metrics class computing complete reports at once — the yardstick every
learner is measured with. Task: both. Pairs with a missing/NaN side are
dropped from the math and reported in n_missing, never hidden.
Metrics.brier_score¶
pythonbrier_score((self, /, y_true, probs))
Brier score of probability predictions. Task: classification.
Args: y_true: targets (positive = > 0.5). probs: predicted probabilities.
Returns: mean squared probability error.
Metrics.calibration¶
pythoncalibration((self, /, y_true, scores, n_bins=10))
Calibration (reliability) bins: mean predicted score vs observed
frequency per bin. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities. n_bins: equal-width bins over [0, 1] (default 10).
Returns: {"bins": [{"left", "right", "mean_score", "observed", "n"}]}
(empty bins omitted; scores clip into edge bins).
Metrics.classification¶
pythonclassification((self, /, y_true, y_pred, probabilities=None))
Complete classification report: accuracy, macro averages, per-class
precision/recall/f1/support, confusion matrix — and with
probabilities (P(class 1)) also logloss, brier and ROC-AUC.
Args: y_true: integer class labels (None/NaN = missing). y_pred: predicted labels (None/NaN = missing). probabilities: P(class 1) per row; enables the probabilistic metrics.
Returns: the full report as a dict.
Raises: ValueError: mismatched lengths, non-integer labels or no observed pairs.
Metrics.classification_by_group¶
pythonclassification_by_group((self, /, keys, y_true, y_pred))
Classification report per group key (sorted by key). Task:
classification.
Args: keys: group label per row (rows with missing key/labels skip). y_true: integer class labels. y_pred: integer class labels.
Returns: {group: classification_report}.
Metrics.classification_scores¶
pythonclassification_scores((self, /, y_true, y_pred))
Single-label classification scorecard (binary or multiclass).
Task: classification.
Args: y_true: integer-like labels. y_pred: integer-like labels.
Returns: {"n", "n_classes", "accuracy", "balanced_accuracy",
"precision_macro/micro/weighted", "recall_*", "f1_*", "mcc",
"jaccard_macro/micro/weighted", "hamming_loss", "zero_one_loss"}.
Metrics.competing_risks¶
pythoncompeting_risks((self, /, durations, events, cif_curves=None, risk_scores=None, times=None))
Complete competing risks report for multi-cause survival endpoints:
cause-specific Harrell's C, Uno's IPCW C, cause-specific IBS and
time-dependent Brier scores over specified or sample decile evaluation horizons.
Args: durations: observed follow-up times. events: event indicators (0 = censored, 1..J = mutually exclusive causes). cif_curves: optional CIF curves mapping {cause: (N, T)} or 3D list (N, n_causes, T). risk_scores: optional risk scores mapping {cause: (N,)} or 2D list (N, n_causes). times: optional evaluation horizon times grid.
Metrics.competing_risks_calibration¶
pythoncompeting_risks_calibration((self, /, durations, events, predicted_cif, horizon))
Competing risks calibration report across causes (Alberge et al. 2026).
Metrics.d_calibration¶
pythond_calibration((self, /, durations, events, surv_at_durations, n_bins=10))
D-Calibration test (Haider et al. 2020) for survival distribution uniformity.
Metrics.decile_table¶
pythondecile_table((self, /, y_true, scores, n_buckets=10))
Decile profile (kds/PerfScoreCard decile table; modelplotpy response at
decile granularity). Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities. n_buckets: equal-count buckets by score descending (default 10).
Returns: [{"decile", "n", "avg_score", "events", "event_rate",
"cum_capture", "cum_event_rate", "lift"}, ...] (decile 1 = top).
Metrics.double_lift¶
pythondouble_lift((self, /, y_true, scores_a, scores_b, n_buckets=10))
Double-lift table: actual vs model A vs model B along A's ranking (CAS
monograph chart; additive vs ratio display is a plotting concern).
Task: regression or classification.
Args: y_true: targets. scores_a: model A predictions (defines the ranking). scores_b: model B predictions. n_buckets: equal-count buckets (default 10).
Returns: [{"bucket", "n", "actual", "pred_a", "pred_b"}, ...].
Metrics.gain_curve¶
pythongain_curve((self, /, y_true, scores))
Cumulative gains curve. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities.
Returns: {"points": [[population_fraction, capture_rate], ...]}
(lift at a point = capture / fraction).
Metrics.hinge_loss¶
pythonhinge_loss((self, /, y_true, decision))
Hinge loss over signed decision margins. Task: classification.
Args: y_true: targets (positive = > 0.5). decision: decision-function margins (signed).
Returns: mean hinge loss.
Metrics.ici¶
pythonici((self, /, durations, events, predicted_survival, horizon))
Alias for integrated_calibration_index.
Metrics.integrated_calibration_index¶
pythonintegrated_calibration_index((self, /, durations, events, predicted_survival, horizon))
Integrated Calibration Index (Austin et al. 2020) at fixed horizon.
Metrics.ks_statistic¶
pythonks_statistic((self, /, y_true, scores))
KS sweep + statistic + Score@KS. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities.
Returns: {"points": [[fpr, tpr, score], ...], "ks", "score_at_ks",
"tpr_at_ks", "fpr_at_ks"}.
Metrics.lift_curve¶
pythonlift_curve((self, /, y_true, scores))
Cumulative lift curve. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities.
Returns: {"points": [[population_fraction, lift], ...]}
(lift = capture / population fraction).
Metrics.log_loss¶
pythonlog_loss((self, /, y_true, probs))
Log loss of probability predictions. Task: classification.
Args: y_true: targets (positive = > 0.5). probs: predicted probabilities.
Returns: mean log loss.
Metrics.multilabel_scores¶
pythonmultilabel_scores((self, /, y_true, y_scores))
Multilabel scorecard: rows are instances, columns are labels, entries
0.5 are positive (raw entries double as ranking scores). Task: multilabel classification.
Args: y_true: label matrix (rows x labels, 0/1). y_scores: prediction/score matrix (same shape).
Returns: {"n", "n_labels", "subset_accuracy", "hamming_loss",
"precision_micro/macro/samples", "recall_*", "f1_*",
"label_ranking_ap", "coverage_error"}.
Metrics.one_calibration¶
pythonone_calibration((self, /, durations, events, predicted_survival, horizon))
1-Calibration slope and intercept (Austin et al. 2020) at fixed horizon.
Metrics.pairwise_distances¶
pythonpairwise_distances((self, /, a, b, metric='euclidean', p=2.0))
Pairwise distances between two row sets. Task: any.
Args: a: first rows (rows x features). b: second rows (same width). metric: "euclidean" | "manhattan" | "chebyshev" | "minkowski" | "cosine" | "correlation" | "hamming" | "jaccard". p: Minkowski power (default 2.0).
Returns: len(a) x len(b) distance matrix.
Metrics.pr_curve¶
pythonpr_curve((self, /, y_true, scores))
Precision-recall curve + average precision. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities.
Returns: {"points": [[recall, precision], ...], "average_precision":
float}.
Metrics.regression¶
pythonregression((self, /, y_true, y_pred))
Complete regression report: n, n_missing, mean_error (actual -
fitted), bias (mean_pred / mean_true), mae, mad, mape (non-zero
targets), rmse, r2, max_error, explained_variance and
deciles — the DAR per-target-decile error profile (the bathtub view).
Args: y_true: actual values (None/NaN = missing). y_pred: fitted values (None/NaN = missing).
Returns: the full report as a dict.
Raises: ValueError: mismatched lengths or no observed pairs.
Metrics.regression_by_group¶
pythonregression_by_group((self, /, keys, y_true, y_pred))
Regression report per group (keys: dates, categories, ...) — the
legacy per-key summary lineage. Rows with a missing key are ignored.
Args: keys: group label per row (None = ignored). y_true: actual values (None/NaN = missing). y_pred: fitted values (None/NaN = missing).
Returns: {group: regression report}, sorted by group.
Raises: ValueError: mismatched lengths or a group with no observed pairs.
Metrics.roc_curve¶
pythonroc_curve((self, /, y_true, scores))
ROC curve + AUC. Task: classification.
Args: y_true: targets (positive = > 0.5). scores: predicted scores/probabilities.
Returns: {"points": [[fpr, tpr], ...], "auc": float}.
Metrics.stability_index¶
pythonstability_index((self, /, reference, current, n_bins=10))
Stability index (PSI on scores, CSI on a feature column). Task:
classification or regression monitoring.
Args: reference: reference-period values (bins derive from their range). current: current-period values. n_bins: equal-width bins (default 10).
Returns: {"psi": float, "bins": [{"left", "right", "reference_share",
"current_share", "contribution"}, ...]}.
Metrics.survival¶
pythonsurvival((self, /, durations, events, risk_scores, surv_curves=None, times=None))
Complete survival analysis report (Harrell C-index, Uno IPCW C-index, Integrated Brier Score).
Args: durations: observed duration / time-to-event per row. events: event indicator (1.0 = event observed, 0.0 = right-censored). risk_scores: predicted risk score (higher score = shorter expected survival). surv_curves: optional individual survival probability curves S(t|x) matrix (N, K). times: optional evaluation horizon times grid.
Metrics.time_series¶
pythontime_series((self, /, y_true, y_pred, y_train=None, seasonal_period=None, std_pred=None))
Complete time series forecasting report (MAE, RMSE, MASE, sMAPE, WAPE, MDA, CRPS).
Args: y_true: actual time series targets. y_pred: predicted values. y_train: optional in-sample training series (used to scale MASE). seasonal_period: optional seasonality m (e.g. 7 for daily, 12 for monthly). std_pred: optional predicted standard deviation per period (used for Gaussian CRPS).
Metrics.uplift¶
pythonuplift((self, /, y_true, treatment, tau_pred))
Complete causal uplift and Qini curve report.
Args: y_true: observed outcome per row. treatment: binary treatment indicator (1.0 = treated, 0.0 = control). tau_pred: predicted treatment effect / uplift score per row.
Returns: dict with ATE, Qini coefficient, normalized Qini, AUUC, decile uplifts, Qini curve.
Metrics.weighted_kappa¶
pythonweighted_kappa((self, /, y_true, y_pred, weights='quadratic'))
Cohen's kappa with optional weights over the label scale. Task:
classification.
Args: y_true: integer-like labels. y_pred: integer-like labels. weights: "none" | "linear" | "quadratic" (default "quadratic").
Returns: kappa (1 = perfect agreement).