Metrics

PREDICT.Metrics.AUPRC(model, outcomeCol='outcome')

LogHook to compute the AUPRC of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the AUPRC of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.AUROC(model, outcomeCol='outcome')

LogHook to compute the AUROC of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the AUROC of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.Accuracy(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the accuracy of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the accuracy of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.CITL(model, outcomeCol='outcome')

LogHook to compute the Calibration-In-The-Large (CITL) of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the CITL of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.CalibrationSlope(model, outcomeCol='outcome')

LogHook to compute the calibration slope of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the calibration slope of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.CoxSnellR2(model, outcomeCol='outcome')

LogHook to compute the Cox and Snell R^2 value of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the pseudo R^2 value of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.F1Score(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the F1 score of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the F1 score of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.OE(model, outcomeCol='outcome')

LogHook to compute the Observed/Expected (O/E) ratio of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the O/E ratio of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.Precision(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the precision of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the precision of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.Recall(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the recall of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the recall of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.Sensitivity(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the sensitivity of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the sensitivity of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.Specificity(model, outcomeCol='outcome', threshold=0.5)

LogHook to compute the specificity of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

  • threshold (float, optional) – Probability threshold at which to classify individuals. Defaults to 0.5.

Returns:

A hook to compute the specificity of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.SumOfDiff(model, outcomeCol='outcome')

LogHook to compute the sum of differences error of a model at each timestep.

Parameters:

  • model (PREDICTModel) – The model to evaluate, must have a predict method.

  • outcomeCol (str, optional) – The column in the dataframe containing the actual outcomes. Defaults to ‘outcome’.

Returns:

A hook to compute the sum of differences error of the model at each timestep when fed data.

Return type:

logHook

PREDICT.Metrics.TrackBayesianCoefs(model)

LogHook to track the Bayesian coefficients of a model at each timestep.

Parameters:

model (BayesianModel) – The model to evaluate, must use the BayesianModel predict method.

Returns:

The name of the hook (‘BayesianCoefficients’), and the current priors for the Bayesian model coefficients.

Return type:

hookname (str), result (float)