Associated with manuscript: Agreement between heuristic shrinkage factor and optimal shrinkage factors in logistic regression for risk prediction: a simulation study across different sample sizes and settings

1 Plots for results from simulation study 1

We present plots of \(\hat{S}_{VH}\) and \(\hat{S}_{boot}\) plotted against \(S_{opt}\) from simulation study 1 for every combination of the simulation inputs:

Figure S1: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = mean(\(\hat{S}_{VH}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S2: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to apparent model performance

Estimator = mean(\(\hat{S}_{VH}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S3: median(\(\hat{S}_{VH}\)) plotted against median(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = median(\(\hat{S}_{VH}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S4: median(\(\hat{S}_{VH}\)) plotted against median(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to apparent model performance

Estimator = median(\(\hat{S}_{VH}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S5: mean(\(\hat{S}_{boot}\)) plotted against mean(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = mean(\(\hat{S}_{boot}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S6: mean(\(\hat{S}_{boot}\)) plotted against mean(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to apparent model performance

Estimator = mean(\(\hat{S}_{boot}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S7: median(\(\hat{S}_{boot}\)) plotted against median(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = median(\(\hat{S}_{boot}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S8: median(\(\hat{S}_{boot}\)) plotted against median(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to apparent model performance

Estimator = median(\(\hat{S}_{boot}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S9: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = mean(\(\hat{S}_{VH}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S10: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), zero covariance in DGM, presented with respect to apparent model performance

Estimator = mean(\(\hat{S}_{VH}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S11: median(\(\hat{S}_{VH}\)) plotted against median(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = median(\(\hat{S}_{VH}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S12: median(\(\hat{S}_{VH}\)) plotted against median(\(S_{opt}\)), zero covariance in DGM, presented with respect to apparent model performance

Estimator = median(\(\hat{S}_{VH}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S13: mean(\(\hat{S}_{boot}\)) plotted against mean(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = mean(\(\hat{S}_{boot}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S14: mean(\(\hat{S}_{boot}\)) plotted against mean(\(S_{opt}\)), zero covariance in DGM, presented with respect to apparent model performance

Estimator = mean(\(\hat{S}_{boot}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S15: median(\(\hat{S}_{boot}\)) plotted against median(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = median(\(\hat{S}_{boot}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S16: median(\(\hat{S}_{boot}\)) plotted against median(\(S_{opt}\)), zero covariance in DGM, presented with respect to apparent model performance

Estimator = median(\(\hat{S}_{boot}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to apparent model performance

Figure S17: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), grouped by mean(\(C_{app}\)), non-zero covariance in DGM

Figure S18: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), grouped by \(C_{pop}\), non-zero covariance in DGM

Figure S19: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), grouped by mean(\(C_{app}\)), zero covariance in DGM

Figure S20: mean(\(\hat{S}_{VH}\)) plotted against mean(\(S_{opt}\)), grouped by \(C_{pop}\), zero covariance in DGM

2 Plots for standard deviation results from simulation study 1

We present plots of the standard deviation of \(\hat{S}_{VH}\) and \(\hat{S}_{boot}\) plotted against the standard deviation of \(S_{opt}\) from simulation study 1. We remind readers we do not expect to see a standard deviation of zero, given that \(S_{opt}\) itself has variation across simulation iterations.

Figure S21: sd(\(\hat{S}_{VH}\)) plotted against sd(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = sd(\(\hat{S}_{VH}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S22: sd(\(\hat{S}_{boot}\)) plotted against sd(\(S_{opt}\)), non-zero covariance in DGM, presented with respect to population-level model performance

Estimator = sd(\(\hat{S}_{boot}\))

Covariance structure = non-zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S23: sd(\(\hat{S}_{VH}\)) plotted against sd(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = sd(\(\hat{S}_{VH}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

Figure S24: sd(\(\hat{S}_{boot}\)) plotted against sd(\(S_{opt}\)), zero covariance in DGM, presented with respect to population-level model performance

Estimator = sd(\(\hat{S}_{boot}\))

Covariance structure = zero covariance in DGM

By variable = presented with respect to population-level model performance

3 Simulation study 2

We present a plot of \(N_{sim}\) against \(N_{original}\):

Figure S25: \(N_{sim}\) plotted against \(N_{original}\)

We also present plots for the instabilty of \(S_{opt}\) in a range of scenarios meeting the sample size criteria for \(N_{original}\) with different sample sizes:

Figure S26: Equivalent of Figure 5 from nanuscript but when criteria is met for \(N_{original}\)