API: Calibrators

Base class

Calibrator

Calibrator(*, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: ABC

Abstract base class for post-hoc segmentation calibrators.

All calibrators share the same interface and tensor convention:

• predictions are channel-first, class axis at dimension 1 ((B, C, *spatial); (N, C) is also accepted);
• targets are integer labels of shape (B, *spatial);
• mask, when given, is (B, *spatial) with True for valid voxels.

Inputs may be raw logits or normalised probs (see input_type). transform and predict_proba always return calibrated probabilities of the same shape as the input.

Parameters:

Name	Type	Description	Default
input_type	str	"logits" (default) if predictions are unnormalised scores, or "probs" if they are probabilities that sum to 1 along the class axis.	'logits'
ignore_index	int	Label value excluded from fitting (default -100).	-100
device	DeviceLike | None	Computation device. None selects CUDA when available, else CPU.	None

Notes

Calibrators fitted by gradient descent (all of them) also accept, as keyword-only constructor arguments, an optional validation-based early stopping rule (Adam only):

• patience -- stop after this many iterations without the validation NLL improving by more than min_delta (default 0.0);
• lr_patience / lr_factor -- multiply the learning rate by lr_factor (default 0.1) after lr_patience iterations without improvement (ReduceLROnPlateau).

Setting either requires val_predictions / val_targets in :meth:fit; the iterate with the best validation NLL is kept. With max_iter acting as an upper bound, this reproduces the "Adam + early stopping on validation NLL" recipe used in the paper.

Source code in src/fiducio/base.py

def __init__(
    self,
    *,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    if input_type not in {"logits", "probs"}:
        raise ValueError(f"input_type must be 'logits' or 'probs', got {input_type!r}")
    self.input_type = input_type
    self.ignore_index = int(ignore_index)
    self.device = resolve_device(device)
    self._num_classes: int | None = None
    self._fitted: bool = False
    self._val: tuple[torch.Tensor, torch.Tensor] | None = None
    self._logger = get_logger(type(self).__name__)

is_fitted property

is_fitted: bool

Whether :meth:fit has been called.

num_classes property

num_classes: int | None

Number of classes seen at fit time, or None before fitting.

fit

fit(predictions: Any, targets: Any, mask: Any | None = None, *, val_predictions: Any | None = None, val_targets: Any | None = None, val_mask: Any | None = None) -> Calibrator

Fit the calibrator on a labelled calibration set.

Parameters:

Name	Type	Description	Default
predictions	Any	(B, C, *spatial) logits or probabilities (see input_type).	required
targets	Any	(B, *spatial) integer labels.	required
mask	Any | None	Optional (B, *spatial) boolean mask; True marks valid voxels.	None
val_predictions	Any | None	Optional held-out validation set, in the same format as predictions / targets / mask. Required when early stopping is configured (patience or lr_patience), and rejected otherwise. The validation NLL (without regularization) is monitored after every Adam step and the best iterate is kept.	None
val_targets	Any | None	Optional held-out validation set, in the same format as predictions / targets / mask. Required when early stopping is configured (patience or lr_patience), and rejected otherwise. The validation NLL (without regularization) is monitored after every Adam step and the best iterate is kept.	None
val_mask	Any | None	Optional held-out validation set, in the same format as predictions / targets / mask. Required when early stopping is configured (patience or lr_patience), and rejected otherwise. The validation NLL (without regularization) is monitored after every Adam step and the best iterate is kept.	None

Notes

Calling fit again on an already-fitted instance re-fits from scratch: all learned parameters are reinitialised and overwritten, and a new num_classes (which may differ from the previous fit) is recorded. No state from the previous fit is reused. If fitting fails, the previous state is preserved. Model outputs are detached from autograd.

Source code in src/fiducio/base.py

def fit(
    self,
    predictions: Any,
    targets: Any,
    mask: Any | None = None,
    *,
    val_predictions: Any | None = None,
    val_targets: Any | None = None,
    val_mask: Any | None = None,
) -> Calibrator:
    """Fit the calibrator on a labelled calibration set.

    Parameters
    ----------
    predictions:
        ``(B, C, *spatial)`` logits or probabilities (see ``input_type``).
    targets:
        ``(B, *spatial)`` integer labels.
    mask:
        Optional ``(B, *spatial)`` boolean mask; ``True`` marks valid voxels.
    val_predictions, val_targets, val_mask:
        Optional held-out validation set, in the same format as
        ``predictions`` / ``targets`` / ``mask``. Required when early
        stopping is configured (``patience`` or ``lr_patience``), and
        rejected otherwise. The validation NLL (without regularization) is
        monitored after every Adam step and the best iterate is kept.

    Notes
    -----
    Calling ``fit`` again on an already-fitted instance re-fits from
    scratch: all learned parameters are reinitialised and overwritten, and
    a new ``num_classes`` (which may differ from the previous fit) is
    recorded. No state from the previous fit is reused. If fitting fails,
    the previous state is preserved. Model outputs are detached from autograd.
    """
    preds, tgts, msk = self._prepare(predictions, targets, mask, with_targets=True)
    assert tgts is not None  # guaranteed by with_targets=True
    num_classes = validate_predictions(preds, input_type=self.input_type)
    validate_targets(
        preds, tgts, msk, num_classes=num_classes, ignore_index=self.ignore_index
    )
    canonical = self._to_canonical(preds)
    z_flat, y_flat = flatten_valid(canonical, tgts, msk, self.ignore_index)
    if z_flat.shape[0] == 0:
        raise ValueError(
            "no valid voxels to fit on (all positions are masked out or equal "
            "ignore_index)"
        )
    val_data = self._prepare_validation(
        val_predictions, val_targets, val_mask, num_classes=num_classes
    )
    candidate = deepcopy(self)
    candidate._num_classes = num_classes
    candidate._val = val_data
    try:
        candidate._fit_core(z_flat, y_flat, num_classes)
    finally:
        candidate._val = None
    candidate._fitted = True
    self.__dict__.update(candidate.__dict__)
    return self

transform

transform(predictions: Any, mask: Any | None = None) -> torch.Tensor

Apply calibration and return probabilities of the input shape.

Parameters:

Name	Type	Description	Default
predictions	Any	(B, C, *spatial) logits or probabilities.	required
mask	Any | None	Optional (B, *spatial) boolean mask. Masked-out voxels are set to 0 across all classes in the output.	None

Source code in src/fiducio/base.py

def transform(self, predictions: Any, mask: Any | None = None) -> torch.Tensor:
    """Apply calibration and return probabilities of the input shape.

    Parameters
    ----------
    predictions:
        ``(B, C, *spatial)`` logits or probabilities.
    mask:
        Optional ``(B, *spatial)`` boolean mask. Masked-out voxels are set to
        0 across all classes in the output.
    """
    if not self._fitted:
        raise NotFittedError("call fit() before transform()")
    logits, msk = self._calibrated_logits(predictions, mask)
    probs = F.softmax(logits, dim=1)
    return apply_mask_to_probabilities(probs, msk)

predict_proba

predict_proba(predictions: Any, mask: Any | None = None) -> torch.Tensor

Alias for :meth:transform; returns calibrated probabilities.

Source code in src/fiducio/base.py

def predict_proba(self, predictions: Any, mask: Any | None = None) -> torch.Tensor:
    """Alias for :meth:`transform`; returns calibrated probabilities."""
    return self.transform(predictions, mask=mask)

decision_function

decision_function(predictions: Any) -> torch.Tensor

Return calibrated logits (pre-softmax) of the input shape.

Unlike :meth:transform, no mask is applied — a logit of 0 is a meaningful value, so masking calibrated logits is left to the caller. softmax of the result along dimension 1 equals :meth:transform.

Source code in src/fiducio/base.py

def decision_function(self, predictions: Any) -> torch.Tensor:
    """Return calibrated **logits** (pre-softmax) of the input shape.

    Unlike :meth:`transform`, no mask is applied — a logit of 0 is a
    meaningful value, so masking calibrated logits is left to the caller.
    ``softmax`` of the result along dimension 1 equals :meth:`transform`.
    """
    if not self._fitted:
        raise NotFittedError("call fit() before decision_function()")
    logits, _ = self._calibrated_logits(predictions, None)
    return logits

fit_transform

fit_transform(predictions: Any, targets: Any, mask: Any | None = None, *, val_predictions: Any | None = None, val_targets: Any | None = None, val_mask: Any | None = None) -> torch.Tensor

Fit on the calibration set, then transform the same predictions.

Source code in src/fiducio/base.py

def fit_transform(
    self,
    predictions: Any,
    targets: Any,
    mask: Any | None = None,
    *,
    val_predictions: Any | None = None,
    val_targets: Any | None = None,
    val_mask: Any | None = None,
) -> torch.Tensor:
    """Fit on the calibration set, then transform the same predictions."""
    self.fit(
        predictions,
        targets,
        mask=mask,
        val_predictions=val_predictions,
        val_targets=val_targets,
        val_mask=val_mask,
    )
    return self.transform(predictions, mask=mask)

save

save(path: Any) -> None

Save this calibrator to path (see :func:fiducio.save_calibrator).

Source code in src/fiducio/base.py

def save(self, path: Any) -> None:
    """Save this calibrator to ``path`` (see :func:`fiducio.save_calibrator`)."""
    from .persistence import save_calibrator

    save_calibrator(self, path)

get_config

get_config() -> dict[str, Any]

Return constructor keyword arguments (excluding device).

Source code in src/fiducio/base.py

def get_config(self) -> dict[str, Any]:
    """Return constructor keyword arguments (excluding ``device``)."""
    config: dict[str, Any] = {
        "input_type": self.input_type,
        "ignore_index": self.ignore_index,
    }
    config.update(self._constructor_config())
    if self._stopping is not None:
        config.update(
            patience=self.patience,
            min_delta=self.min_delta,
            lr_patience=self.lr_patience,
            lr_factor=self.lr_factor,
        )
    return config

Temperature scaling

TemperatureScaling

TemperatureScaling(*, init_temperature: float = 1.0, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: Calibrator

Temperature scaling: softmax(z / T) with a single scalar T > 0.

The simplest and most robust post-hoc calibrator. It cannot change the predicted class ordering; it only rescales confidence.

Parameters:

Name	Type	Description	Default
init_temperature	float	Initial temperature (must be positive).	1.0
optimizer	str	"adam" (default) or "lbfgs".	'adam'
lr	float | None	Learning rate. Defaults to 0.1 (Adam) or 1.0 (L-BFGS).	None
max_iter	int | None	Maximum optimizer iterations. Defaults to 200 (Adam) or 100 (L-BFGS).	None
patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
min_delta	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_factor	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
input_type	str	See :class:fiducio.Calibrator.	'logits'
ignore_index	str	See :class:fiducio.Calibrator.	'logits'
device	str	See :class:fiducio.Calibrator.	'logits'

References

Guo et al. (2017), On Calibration of Modern Neural Networks, ICML.

Source code in src/fiducio/calibrators/temperature.py

def __init__(
    self,
    *,
    init_temperature: float = 1.0,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.init_temperature = positive_finite(init_temperature, "init_temperature")
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.temperature: float = float(init_temperature)

Ensemble temperature scaling

EnsembleTemperatureScaling

EnsembleTemperatureScaling(*, init_temperature: float = 1.0, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: Calibrator

Ensemble Temperature Scaling.

Calibrated probabilities are a convex combination of a temperature-scaled distribution, the original distribution and the uniform distribution:

.. math:: p = w_0\,\mathrm{softmax}(z/T) + w_1\,\mathrm{softmax}(z) + w_2\,u

where :math:w lies on the 3-simplex and :math:u is uniform. Fitting is done in two stages: first the temperature T (NLL), then the mixture weights w with T fixed.

Parameters:

Name	Type	Description	Default
init_temperature	float	Initial temperature for stage 1.	1.0
optimizer	str	"adam" (default) or "lbfgs".	'adam'
lr	float | None	Learning rate. Defaults to 0.1 (Adam) or 1.0 (L-BFGS).	None
max_iter	int | None	Maximum optimizer iterations per stage. Defaults to 200 (Adam) or 100 (L-BFGS).	None
patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
min_delta	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_factor	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
input_type	str	See :class:fiducio.Calibrator.	'logits'
ignore_index	str	See :class:fiducio.Calibrator.	'logits'
device	str	See :class:fiducio.Calibrator.	'logits'

References

Zhang et al. (2020), Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning, ICML.

Source code in src/fiducio/calibrators/ensemble_temperature.py

def __init__(
    self,
    *,
    init_temperature: float = 1.0,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.init_temperature = positive_finite(init_temperature, "init_temperature")
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.temperature: float = float(init_temperature)
    self.weights: torch.Tensor = torch.tensor([1.0, 0.0, 0.0], device=self.device)

Vector scaling

VectorScaling

VectorScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _AffineCalibrator

Vector scaling: softmax(diag(w) z + b).

A per-class generalisation of temperature scaling with one scale w_c and one bias b_c per class. Operates on logits (or log of probabilities when input_type='probs').

Parameters:

Name	Type	Description	Default
optimizer	str	"adam" (default) or "lbfgs".	'adam'
lr	float | None	Learning rate. Defaults to 0.1 (Adam) or 1.0 (L-BFGS).	None
max_iter	int | None	Maximum optimizer iterations. Defaults to 200 (Adam) or 100 (L-BFGS).	None
lambda_reg	float	L2 penalty pulling the scale vector towards 1.	0.0
mu_reg	float	L2 penalty on the bias vector.	0.0
patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
min_delta	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_factor	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
input_type	str	See :class:fiducio.Calibrator.	'logits'
ignore_index	str	See :class:fiducio.Calibrator.	'logits'
device	str	See :class:fiducio.Calibrator.	'logits'

References

Guo et al. (2017), On Calibration of Modern Neural Networks, ICML.

Source code in src/fiducio/calibrators/_affine.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self._weight: torch.Tensor | None = None  # (C, C) or (C,) for diagonal
    self._bias: torch.Tensor | None = None  # (C,)
    self._row_sum: torch.Tensor | None = None

Matrix scaling

MatrixScaling

MatrixScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _AffineCalibrator

Matrix scaling: softmax(W z + b) with a full C x C matrix.

Optional off-diagonal / bias L2 regularisation (ODIR) penalizes class mixing and bias. Diagonal entries are not directly penalized.

Parameters:

Name	Type	Description	Default
optimizer	str	"adam" (default) or "lbfgs".	'adam'
lr	float | None	Learning rate. Defaults to 0.1 (Adam) or 1.0 (L-BFGS).	None
max_iter	int | None	Maximum optimizer iterations. Defaults to 200 (Adam) or 100 (L-BFGS).	None
lambda_reg	float	L2 penalty on the off-diagonal entries of W (ODIR).	0.0
mu_reg	float	L2 penalty on the bias vector (ODIR).	0.0
patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
min_delta	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_factor	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
input_type	str	See :class:fiducio.Calibrator.	'logits'
ignore_index	str	See :class:fiducio.Calibrator.	'logits'
device	str	See :class:fiducio.Calibrator.	'logits'

References

Guo et al. (2017), On Calibration of Modern Neural Networks, ICML; Kull et al. (2019) for off-diagonal/intercept regularisation.

Source code in src/fiducio/calibrators/_affine.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self._weight: torch.Tensor | None = None  # (C, C) or (C,) for diagonal
    self._bias: torch.Tensor | None = None  # (C,)
    self._row_sum: torch.Tensor | None = None

Translation-invariant matrix scaling

TranslationInvariantMatrixScaling

TranslationInvariantMatrixScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _AffineCalibrator

Constrained matrix scaling that is invariant to logit translations.

The matrix W is constrained so that every row has the same learned sum. Because softmax ignores constant shifts of its input, this makes the calibrated output invariant to adding the same constant to every input logit (g(z + c·1) = g(z)). The first C-1 columns and the common row sum are optimized, with the final column reconstructed. Initialization is the identity and regularization uses the reconstructed matrix, matching the research MSc parameterization.

Parameters are identical to :class:MatrixScaling.

Source code in src/fiducio/calibrators/_affine.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self._weight: torch.Tensor | None = None  # (C, C) or (C,) for diagonal
    self._bias: torch.Tensor | None = None  # (C,)
    self._row_sum: torch.Tensor | None = None

Dirichlet calibration

DirichletCalibration

DirichletCalibration(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _AffineCalibrator

Dirichlet calibration: softmax(W log p + b).

A log-linear transform in probability space, equivalent to matrix scaling applied to log-probabilities. When input_type='logits' the inputs are converted to log-probabilities with log_softmax before calibration. Off-diagonal / bias L2 regularisation (ODIR) is recommended.

Parameters:

Name	Type	Description	Default
optimizer	str	"adam" (default) or "lbfgs".	'adam'
lr	float | None	Learning rate. Defaults to 0.1 (Adam) or 1.0 (L-BFGS).	None
max_iter	int | None	Maximum optimizer iterations. Defaults to 200 (Adam) or 100 (L-BFGS).	None
lambda_reg	float	L2 penalty on the off-diagonal entries of W (ODIR).	0.0
mu_reg	float	L2 penalty on the bias vector (ODIR).	0.0
patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
min_delta	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_patience	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
lr_factor	int | None	Optional validation-based early stopping (Adam only), see :class:fiducio.Calibrator. fit then requires val_predictions and val_targets.	None
input_type	str	See :class:fiducio.Calibrator.	'logits'
ignore_index	str	See :class:fiducio.Calibrator.	'logits'
device	str	See :class:fiducio.Calibrator.	'logits'

References

Kull et al. (2019), Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration, NeurIPS.

Source code in src/fiducio/calibrators/_affine.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=0.1, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self._weight: torch.Tensor | None = None  # (C, C) or (C,) for diagonal
    self._bias: torch.Tensor | None = None  # (C,)
    self._row_sum: torch.Tensor | None = None

Class-conditional matrix scaling

ClassConditionalMatrixScaling

ClassConditionalMatrixScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, independent_experts: bool = False, init_alpha: float = 1.0, init_floor: float = 1e-06, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _ClassConditionalBase

Class-conditional matrix scaling (CMS).

One unconstrained affine map A_c log p + b_c per uncalibrated top class c. Unlike the preserving variants it may change the argmax. Off-diagonal and bias L2 regularization (lambda_reg / mu_reg) penalize class mixing and bias, leaving the diagonal unpenalized. Experts are optimized jointly by default; set independent_experts=True to fit each one in its own optimization loop.

Source code in src/fiducio/calibrators/class_conditional.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    independent_experts: bool = False,
    init_alpha: float = 1.0,
    init_floor: float = 1e-6,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=1e-2, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self.independent_experts = bool(independent_experts)
    self.init_alpha = positive_finite(init_alpha, "init_alpha")
    self.init_floor = positive_finite(init_floor, "init_floor")
    self._raw_b: torch.Tensor | None = None  # (C, K, K) or (C, C, C)
    self._raw_mu: torch.Tensor | None = None  # (C, K) or (C, C)

Argmax-preserving matrix scaling

ArgmaxPreservingMatrixScaling

ArgmaxPreservingMatrixScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, independent_experts: bool = False, init_alpha: float = 1.0, init_floor: float = 1e-06, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _ClassConditionalBase

Argmax-preserving class-conditional matrix scaling (CMSAP).

Parameterised through non-negative margins between the top class and its competitors, which guarantees the calibrated argmax equals the uncalibrated argmax for every voxel. Experts are optimized jointly by default; set independent_experts=True to fit each one in its own optimization loop.

Source code in src/fiducio/calibrators/class_conditional.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    independent_experts: bool = False,
    init_alpha: float = 1.0,
    init_floor: float = 1e-6,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=1e-2, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self.independent_experts = bool(independent_experts)
    self.init_alpha = positive_finite(init_alpha, "init_alpha")
    self.init_floor = positive_finite(init_floor, "init_floor")
    self._raw_b: torch.Tensor | None = None  # (C, K, K) or (C, C, C)
    self._raw_mu: torch.Tensor | None = None  # (C, K) or (C, C)

Order-preserving matrix scaling

OrderPreservingMatrixScaling

OrderPreservingMatrixScaling(*, optimizer: str = 'adam', lr: float | None = None, max_iter: int | None = None, patience: int | None = None, min_delta: float = 0.0, lr_patience: int | None = None, lr_factor: float = 0.1, lambda_reg: float = 0.0, mu_reg: float = 0.0, independent_experts: bool = False, init_alpha: float = 1.0, init_floor: float = 1e-06, input_type: str = 'logits', ignore_index: int = -100, device: DeviceLike | None = None)

Bases: _ClassConditionalBase

Order-preserving class-conditional matrix scaling (CMSOP).

Parameterised through non-negative gaps between consecutively ranked classes, which guarantees the full class ranking is preserved for every voxel. Experts are optimized jointly by default; set independent_experts=True to fit each one in its own optimization loop.

Source code in src/fiducio/calibrators/class_conditional.py

def __init__(
    self,
    *,
    optimizer: str = "adam",
    lr: float | None = None,
    max_iter: int | None = None,
    patience: int | None = None,
    min_delta: float = 0.0,
    lr_patience: int | None = None,
    lr_factor: float = 0.1,
    lambda_reg: float = 0.0,
    mu_reg: float = 0.0,
    independent_experts: bool = False,
    init_alpha: float = 1.0,
    init_floor: float = 1e-6,
    input_type: str = "logits",
    ignore_index: int = -100,
    device: DeviceLike | None = None,
) -> None:
    super().__init__(input_type=input_type, ignore_index=ignore_index, device=device)
    self.optimizer, self.lr, self.max_iter = resolve_optimizer(
        optimizer, lr, max_iter,
        adam_lr=1e-2, lbfgs_lr=1.0, adam_max_iter=200, lbfgs_max_iter=100,
    )
    self._init_stopping(self.optimizer, patience, min_delta, lr_patience, lr_factor)
    self.lambda_reg = positive_finite(lambda_reg, "lambda_reg", allow_zero=True)
    self.mu_reg = positive_finite(mu_reg, "mu_reg", allow_zero=True)
    self.independent_experts = bool(independent_experts)
    self.init_alpha = positive_finite(init_alpha, "init_alpha")
    self.init_floor = positive_finite(init_floor, "init_floor")
    self._raw_b: torch.Tensor | None = None  # (C, K, K) or (C, C, C)
    self._raw_mu: torch.Tensor | None = None  # (C, K) or (C, C)