A threshold-free summary index of prediction accuracy for censored time to event data

Yan Yuan, Qian M. Zhou, Bingying Li, Hengrui Cai, Eric J. Chow, Gregory Armstrong

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores to screen asymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecified future time t 0 . Accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cutoff value created barriers for practitioners and researchers. In this paper, we propose a threshold-free summary index of positive predictive values that accommodates time-dependent event status and competing risks. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between 2 risk scores for censored time to event data. We conduct a simulation study to examine the finite-sample performance of the proposed estimation and inference procedures. Lastly, we illustrate the use of this measure on a real data example, comparing 2 risk score systems for predicting heart failure in childhood cancer survivors.

Original languageEnglish (US)
Pages (from-to)1671-1681
Number of pages11
JournalStatistics in Medicine
Volume37
Issue number10
DOIs
StatePublished - May 10 2018

Fingerprint

Prediction
Heart Failure
Competing Risks
Nonparametric Estimator
Performance Prediction
Preventive Medicine
Threshold Value
Scoring
Cancer
Simulation Study
Research Personnel
Predict
Population
Neoplasms

All Science Journal Classification (ASJC) codes

  • Epidemiology
  • Statistics and Probability

Cite this

A threshold-free summary index of prediction accuracy for censored time to event data. / Yuan, Yan; Zhou, Qian M.; Li, Bingying; Cai, Hengrui; Chow, Eric J.; Armstrong, Gregory.

In: Statistics in Medicine, Vol. 37, No. 10, 10.05.2018, p. 1671-1681.

Research output: Contribution to journalArticle

Yuan, Yan ; Zhou, Qian M. ; Li, Bingying ; Cai, Hengrui ; Chow, Eric J. ; Armstrong, Gregory. / A threshold-free summary index of prediction accuracy for censored time to event data. In: Statistics in Medicine. 2018 ; Vol. 37, No. 10. pp. 1671-1681.
@article{06dc0b58dac7472e9beedbb896c2f4ab,
title = "A threshold-free summary index of prediction accuracy for censored time to event data",
abstract = "Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores to screen asymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecified future time t 0 . Accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cutoff value created barriers for practitioners and researchers. In this paper, we propose a threshold-free summary index of positive predictive values that accommodates time-dependent event status and competing risks. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between 2 risk scores for censored time to event data. We conduct a simulation study to examine the finite-sample performance of the proposed estimation and inference procedures. Lastly, we illustrate the use of this measure on a real data example, comparing 2 risk score systems for predicting heart failure in childhood cancer survivors.",
author = "Yan Yuan and Zhou, {Qian M.} and Bingying Li and Hengrui Cai and Chow, {Eric J.} and Gregory Armstrong",
year = "2018",
month = "5",
day = "10",
doi = "10.1002/sim.7606",
language = "English (US)",
volume = "37",
pages = "1671--1681",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "John Wiley and Sons Ltd",
number = "10",

}

TY - JOUR

T1 - A threshold-free summary index of prediction accuracy for censored time to event data

AU - Yuan, Yan

AU - Zhou, Qian M.

AU - Li, Bingying

AU - Cai, Hengrui

AU - Chow, Eric J.

AU - Armstrong, Gregory

PY - 2018/5/10

Y1 - 2018/5/10

N2 - Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores to screen asymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecified future time t 0 . Accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cutoff value created barriers for practitioners and researchers. In this paper, we propose a threshold-free summary index of positive predictive values that accommodates time-dependent event status and competing risks. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between 2 risk scores for censored time to event data. We conduct a simulation study to examine the finite-sample performance of the proposed estimation and inference procedures. Lastly, we illustrate the use of this measure on a real data example, comparing 2 risk score systems for predicting heart failure in childhood cancer survivors.

AB - Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores to screen asymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecified future time t 0 . Accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cutoff value created barriers for practitioners and researchers. In this paper, we propose a threshold-free summary index of positive predictive values that accommodates time-dependent event status and competing risks. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between 2 risk scores for censored time to event data. We conduct a simulation study to examine the finite-sample performance of the proposed estimation and inference procedures. Lastly, we illustrate the use of this measure on a real data example, comparing 2 risk score systems for predicting heart failure in childhood cancer survivors.

UR - http://www.scopus.com/inward/record.url?scp=85041732865&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041732865&partnerID=8YFLogxK

U2 - 10.1002/sim.7606

DO - 10.1002/sim.7606

M3 - Article

VL - 37

SP - 1671

EP - 1681

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 10

ER -