Calibrators

All calibrators share the same API and tensor convention (see Quickstart). They differ in how expressive the learned transform is and in what they guarantee.

Available calibrators

Class	Alias	Parameters	Translation-invariant	Decision preservation
TemperatureScaling	TS	one scalar T	yes	argmax and full order
EnsembleTemperatureScaling	ETS	T + 3 mixture weights	yes	argmax and full order
VectorScaling	VS	per-class scale + bias	no	—
MatrixScaling	MS	full C × C matrix + bias	no	—
TranslationInvariantMatrixScaling	MSc	constrained C × C matrix + bias	yes	—
DirichletCalibration	DC	C × C matrix on log-probs	yes	—
ClassConditionalMatrixScaling	CMS / CDC	one affine map per top class	yes	—
ArgmaxPreservingMatrixScaling	CMSAP / CMSap	per-class margin map	yes	argmax
OrderPreservingMatrixScaling	CMSOP / CMSop	per-class gap map	yes	argmax and full order
BinaryIsotonicCalibration	—	up to n_bins fitted points	yes	—

Translation-invariant means the output is unchanged when the same constant is added to every input logit of a voxel. Short aliases (TS, MS, CMS, ...) are provided for convenience; explicit names are easier to read.

The table describes logit inputs. With input_type="probs", an upstream logit shift has already been removed by softmax.

The ETS guarantee holds in exact arithmetic — a non-negative weighted sum of monotone maps preserves the order — but not bit-for-bit in float32 once the mixture collapses towards the uniform component: ties and near-ties (top-2 gaps below the float32 resolution) can then resolve differently. Treat ETS as order-preserving up to floating-point rounding.

Binary isotonic calibration and MHR

BinaryIsotonicCalibration fits a nondecreasing positive-class map with weighted PAVA on at most n_bins bins (default 250). It accepts two-channel probabilities or logits and hard or MHR targets. This approach is inspired by TwinTrack (Kirscher et al., MIDL 2026, short paper). Tie handling and interpolation are explicit Fiducio conventions; exact reproduction has not been verified. It can change a segmentation at threshold 0.5, and has no optimizer or early stopping. The nine parametric calibrators also accept MHR distributions. See multi-rater calibration for details and metric conventions.

Choose fitting n_bins on validation cases held out from the fit. The fitting bins and the evaluation bins are separate settings:

from fiducio import BinaryIsotonicCalibration, expected_calibration_error

# cal_* and val_* come from disjoint case splits.
candidates = []
for n_bins in (16, 64, 250):  # fitting bins
    cal = BinaryIsotonicCalibration(input_type="probs", n_bins=n_bins, device="cpu").fit(
        cal_probabilities, cal_mhr, mask=cal_annotated
    )
    ece = expected_calibration_error(
        cal.transform(val_probabilities, mask=val_annotated),
        val_mhr,
        mask=val_annotated,
        class_index=1,
        n_bins=50,  # evaluation bins
    )
    candidates.append((ece, cal))
validation_ece, calibrator = min(candidates, key=lambda item: item[0])

Evaluate the selected calibrator once on separate test cases. Ties can reduce the actual number of fitting bins. A small high-confidence tail may be merged into the final bin, so extreme scores can be pulled towards that bin's mean.

Decision and ordering guarantees in the table hold in exact arithmetic and are subject to floating-point ties and rounding.

Paper method mapping

The following methods of Rethinking Post-Hoc Calibration in Semantic Segmentation (Kirscher et al., Transactions on Machine Learning Research, 2026; preprint arXiv:2607.01902) are implemented here. LTS is not included in this release; this table is not the complete experimental method inventory.

Paper method	Class	Alias
TS	TemperatureScaling	TS
MS	MatrixScaling	MS
MSc	TranslationInvariantMatrixScaling	MSc
CDC	ClassConditionalMatrixScaling	CDC
CMSap	ArgmaxPreservingMatrixScaling	CMSap
CMSop	OrderPreservingMatrixScaling	CMSop

Class-conditional calibrators

CDC, CMSap and CMSop route each voxel to an expert affine map selected by the uncalibrated top class. By default (independent_experts=False, matching the paper) all experts are optimized jointly — a single optimizer minimizes one cross-entropy loss over every voxel at once, with each voxel passing through its own expert's map. lambda_reg and mu_reg regularize the affine map induced in the common logit space (not the raw per-expert parameters), averaged across experts. This logit-space regularization makes the penalty meaningful for CMSap/CMSop, whose raw parameters are non-negative margins/gaps rather than matrix entries; a penalty on the raw parameters would not correspond to "close to the identity" in the space the map operates in.

Set independent_experts=True to fit each expert in its own optimization loop, using only the voxels routed to it. This avoids an expert with few routed voxels being drowned out by the joint loss, at the cost of C times the optimizer work (one full max_iter-step fit per class instead of one shared fit).

The preserving variants (ArgmaxPreservingMatrixScaling, OrderPreservingMatrixScaling) also accept init_alpha (default 1.0) and init_floor (default 1e-6): their positive margin/gap parameters start at init_floor, except the diagonal entries, which start at init_alpha. These are constructor-only starting values, not learned parameters; the unconstrained ClassConditionalMatrixScaling accepts them but does not use them.

Their parameters grow as O(C^3), so they suit low-to-moderate class counts, and are the most expressive option: they benefit most from a reasonably sized calibration set.

Choosing a calibrator

• Start with TemperatureScaling. It has a single parameter, is very hard to overfit, preserves the argmax up to floating-point ties and rounding, and is a useful baseline for over- or under-confidence.
• Need per-class flexibility? Try VectorScaling, then MatrixScaling. Use the off-diagonal/bias regularisation (lambda_reg, mu_reg) when the number of classes is large relative to the calibration set, to avoid overfitting.
• Want matrix scaling invariant to logit shifts? Use TranslationInvariantMatrixScaling.
• Probability-space transform? DirichletCalibration applies an affine map to log-probabilities.
• Calibration must not change the predicted labels? Use ArgmaxPreservingMatrixScaling: the calibrated argmax equals the uncalibrated one at every valid voxel up to floating-point ties, so an argmax segmentation, and Dice/IoU computed from it, is unchanged. Thresholded or post-processed segmentations are not covered.
• Calibration must not change the class ranking at all? Use OrderPreservingMatrixScaling, which preserves the full per-voxel ordering.
• Want a flexible per-region map with no constraints? ClassConditionalMatrixScaling fits one affine map per uncalibrated top class.
• Only two classes? MSc is very restrictive for C = 2; the paper finds it significantly worse on BA-ECE. Prefer a less constrained calibrator.
• Binary problem with a flexible monotone map? BinaryIsotonicCalibration fits a nondecreasing, piecewise-linear map on equal-mass bins. Unlike the parametric maps it can pull extreme scores toward the last bin mean when the top-confidence voxels weigh less than one bin; tune n_bins for your data.

Regularisation

MatrixScaling, TranslationInvariantMatrixScaling, DirichletCalibration and the class-conditional calibrators accept:

• lambda_reg — L2 penalty on off-diagonal matrix entries (diagonal entries are not directly penalized);
• mu_reg — L2 penalty on the bias.

The penalty is lambda_reg * mean(off-diagonal W^2) + mu_reg * mean(b^2) (the mean is taken over experts for the class-conditional family). Both terms are means, matching Kull et al.'s (2019) normalized ODIR (their code with reg_norm=True); values from un-normalized settings are not transferable.

VectorScaling's lambda_reg pulls the scale vector towards 1 and its mu_reg penalizes the bias. Start at 0 and increase if the calibrator overfits a small calibration set.

MSc optimizes C-1 free columns and a learned common row sum; the final column is reconstructed before applying the off-diagonal penalty. Initialization is the identity. This differs from the older pre-release row-centering approach.

The guarantees above apply to valid unmasked voxels, subject to floating-point precision. Exact ties follow PyTorch selection/sorting behavior; preserving strict ranks does not imply preserving every set of tied scores.