unreliability index U

You replied on Wed 2021-02-10 11:44
OShea, Daniel T (DNR) <daniel.oshea@state.mn.us>
Wed 2021-02-10 09:29

Frank (Dr. Harrell),

 

I have been refining my process for externally validating logistic regression models (resource selection functions).  For calibration, I use both visual examination of calibration curves and a test for calibration (alpha=1 and beta =1) mainly the unreliability index with p value from val.prob.  I have also explored some additional R packages and came across the givitiR package and their calibration test and calibration belt (Finazzi et al 2011, Nattino et al. 2017) .  Their test provides the same test statistic and p-value as your U index.  In givitiR one must specify that external data is being used to be equivalent to the U index.

 

I just find this funny and am wondering is this the same test or somehow by chance a different test results in same information.  Unfortunately, I do not have the math skills to dissect each test.

 

Dan