Integrated Brier score times problem

[tomas-silveira]

Hello again sebp, I have a question regarding the calculation of the integrated brier score. Following your tutorial, you use a data-driven approach to get which time points to compute the brier score, which in your case, are the points between the 10th and 90th percentile of the test set.
However, in my case I have a highly imbalanced dataset, with very few unique events. In this case, if I use a percentile interval like in your example, I get the same upper and lower bounds. My question is, what is the criteria that one should use to get the times array, such that we have a 'well computed' brier score? Thank you again!

[sebp]

I'm not aware of a general rule. Often, you can decide on the time points based on your application. For instance, if 30 days is an important cut-off for your application, it would make sense to evaluate it. If you have very few time unique points, it might not make sense to compute an integrated Brier score, but just compute the regular Brier score at one particular time point.