Tests

PREDICT

class tests.test_predict.MockModel

Bases: object

Mock model for testing the PREDICT class.

tests.test_predict.predict_instance(sample_data)

Creates an instance of the PREDICT class using the provided sample data.

Parameters:

sample_data (pd.DataFrame) – Sample data to be used for creating the PREDICT instance.

Returns:

An instance of the PREDICT class.

tests.test_predict.sample_data()

Creates a sample dataset for testing the PREDICT class.

Returns:

Fake dataset with date and value columns.

Return type:

pd.DataFrame

tests.test_predict.test_add_log(predict_instance)

Test that logs can be added to the log dictionary.

Parameters:

• predict_instance (class) – Initialises with a dataset and a prediction model.

• logs (Manages prediction windows and)

• functions (allows for the addition of hooks to execute)

• process. (during the logging)

tests.test_predict.test_add_log_hook(predict_instance)

Test that hooks can be added to the log.

Parameters:

• predict_instance (class) – Initialises with a dataset and a prediction model.

• logs (Manages prediction windows and)

• functions (allows for the addition of hooks to execute)

• process. (during the logging)

tests.test_predict.test_get_log(predict_instance)

Test that the getLog method returns the log dictionary.

Parameters:

• predict_instance (class) – Initialises with a dataset and a prediction model.

• logs (Manages prediction windows and)

• functions (allows for the addition of hooks to execute)

• process. (during the logging)

tests.test_predict.test_initialisation(predict_instance)

Test that the PREDICT class is initialised correctly.

Parameters:

• predict_instance (class) – Initialises with a dataset and a prediction model.

• logs (Manages prediction windows and)

• functions (allows for the addition of hooks to execute)

• process. (during the logging)

tests.test_predict.test_run(predict_instance)

Test that the run method executes the prediction model.

Parameters:

• predict_instance (class) – Initialises with a dataset and a prediction model.

• logs (Manages prediction windows and)

• functions (allows for the addition of hooks to execute)

• process. (during the logging)

Metrics

class tests.test_metrics.MockModel

Bases: object

Mock model class for testing.

tests.test_metrics.test_CITL_calculation()

Tests the computation of the calibration-in-the-large metric on dummy data of a perfect model.

tests.test_metrics.test_CITL_calculation2()

Tests the computation of the calibration-in-the-large metric on dummy data compared to a result from R.

tests.test_metrics.test_CITL_calculation3()

Tests the computation of the calibration-in-the-large metric on a model trained on the heart disease dataset.

tests.test_metrics.test_OE_calculation()

Tests the computation of the O/E metric for a perfect model on dummy data.

tests.test_metrics.test_OE_calculation2()

Tests the computation of the O/E metric on dummy data with the result compared to an R result.

tests.test_metrics.test_OE_calculation3()

Tests the computation of the O/E metric on a model trained on the heart disease dataset.

tests.test_metrics.test_accuracy_computation()

Tests the computation of the accuracy metric on dummy data assuming a perfect model.

tests.test_metrics.test_accuracy_computation2()

Tests the computation of the accuracy metric on a model trained on the heart disease dataset.

tests.test_metrics.test_auprc_computation()

Tests the computation of the AUPRC metric on dummy data assuming a perfect model.

tests.test_metrics.test_auprc_computation2()

Tests the computation of the AUPRC metric on a model trained on the heart disease dataset.

tests.test_metrics.test_auroc_computation()

Tests the computation of the AUROC metric on dummy data assuming a perfect model.

tests.test_metrics.test_auroc_computation2()

Tests the computation of the AUROC metric on a model trained on the heart disease dataset.

tests.test_metrics.test_c_slope()

Tests the computation of the calibration slope metric on a model trained on the heart disease dataset.

tests.test_metrics.test_c_slope_to_r()

Tests the computation of the calibration slope metric on dummy data compared to the results on the same data using R.

tests.test_metrics.test_c_slope_to_r2()

Tests the computation of the calibration slope metric on dummy data compared to a result from R.

tests.test_metrics.test_coxsnell_R2_calculation()

Tests the computation of the Cox-Snell R^2 metric for an imperfect model and a model predicting all the same probabilities on dummy data.

tests.test_metrics.test_coxsnell_R2_calculation2()

Tests the computation of the Cox-Snell R^2 metric on a model trained on the heart disease dataset.

tests.test_metrics.test_f1_computation()

Tests the computation of the F1 score metric on dummy data assuming a perfect model.

tests.test_metrics.test_f1_computation2()

Tests the computation of the F1 score metric on a model trained on the heart disease dataset.

tests.test_metrics.test_precision_computation()

Tests the computation of the precision metric on dummy data assuming a perfect model.

tests.test_metrics.test_precision_computation2()

Tests the computation of the precision metric on a model trained on the heart disease dataset.

tests.test_metrics.test_recall_computation()

Tests the computation of the recall metric on dummy data assuming a perfect model.

tests.test_metrics.test_recall_computation2()

Tests the computation of the recall metric on a model trained on the heart disease dataset.

tests.test_metrics.test_sensitivity_computation()

Tests the computation of the sensitivity metric on dummy data assuming a perfect model.

tests.test_metrics.test_sensitivity_computation2()

Tests the computation of the sensitivity metric on a model trained on the heart disease dataset.

tests.test_metrics.test_specificity_computation()

Tests the computation of the specificity metric on dummy data assuming a perfect model.

tests.test_metrics.test_specificity_computation2()

Tests the computation of the specificity metric on a model trained on the heart disease dataset.

tests.test_metrics.test_sum_of_diff_calculation()

Tests the computation of the logistic regression error metric on dummy data assuming a perfect model.

tests.test_metrics.test_sum_of_diff_calculation2()

Tests the computation of the logistic regression error metric on dummy data assuming a model that gets predictions right 25% of the time.

tests.test_metrics.test_sum_of_diff_calculation3()

Tests the computation of the logistic regression error metric on dummy data assuming a model that never gets predictions right.

tests.test_metrics.test_track_bayesian_coefs(mock_model)

Tests the TrackBayesianCoefs hook by checking if the coefficients remain the same.

Models

tests.test_models.test_add_hooks()

Test that hooks can be added to the model.

tests.test_models.test_evaluate_predictions()

Test that the evaluate predictions model works.

tests.test_models.test_recalibrate_predictions()

Test that the recalibrate predictions model produces a non-None output and checking the recalibration hook is added.

tests.test_models.test_regular_recalibrations()

Test that the regular recalibrations model updates when expected.

Triggers

class tests.test_triggers.MockModel(preds)

Bases: object

Mock model class for testing

tests.test_triggers.input_data()

Create dummy binary outcome data for testing.

Returns:

Dummy binary outcome data.

Return type:

pd.DataFrame

tests.test_triggers.test_accuracy_above_threshold(input_data)

Test that the accuracy threshold trigger works when accuracy is above the threshold.

Parameters:

input_data (pd.DataFrame) – Dummy binary outcome data.

tests.test_triggers.test_accuracy_below_threshold(input_data)

Test that the accuracy threshold trigger works when accuracy is below the threshold.

Parameters:

input_data (pd.DataFrame) – Dummy binary outcome data.

tests.test_triggers.test_accuracy_same_as_threshold(input_data)

Test that the accuracy threshold trigger works when accuracy is equal to the threshold.

Parameters:

input_data (pd.DataFrame) – Dummy binary outcome data.

tests.test_triggers.test_bayesian_refit_trigger(sample_data)

Test the Bayesian refit trigger function.

Parameters:

sample_data (DataFrame) – Sample datetime data for testing.