A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods

Gary S. Collins, Omar Omar, Milensu Shanyinde, Ly Mee Yu

Research output: Contribution to journalReview article

70 Citations (Scopus)

Abstract

Background: Chronic kidney disease (CKD) is a global health concern that is increasing mainly as the result of increasing incidences of diabetes and hypertension. Furthermore, if left untreated, individuals with CKD may progress to end-stage kidney failure. Identifying individuals with undiagnosed CKD or those who are at an increased risk of developing CKD or progressing to end-stage kidney disease (ESKD) is therefore an important challenge. We sought to systematically review and critically assess the conduct and reporting of methods used to develop risk prediction models for predicting the risk of having undiagnosed (prevalent) or future risk of developing (incident) CKD or end-stage kidney failure in adults. Methods: We conducted a systematic search of PubMed database to identify studies published up until September 2011 that describe the development of models combining two or more variables to predict the risk of prevalent or incident CKD or ESKD. We extracted key information that describes aspects of developing a prediction model, including the study design, data quality, sample size and number of events, outcome definition, risk predictor selection and coding, missing data, model-building strategies, and aspects of performance. Results: Eleven studies describing the development of 14 prediction models were included. Eight studies reported the development of 11 models to predict incident CKD or ESKD, whereas 3 studies developed models for prevalent CKD. A total of 97 candidate risk predictors were considered, and 43 different risk predictors featured in the 14 prediction models. A method, not recommended to select risk predictors for inclusion in the multivariate model, using statistical significance from univariate screening was carried out in six studies. Missing data were frequently poorly handled and reported with no mention of missing data in four studies; 4 studies explicitly excluded individuals with missing data, and only 2 studies used multiple imputation to replace missing values. Conclusion: We found that prediction models for chronic kidney were often developed using inappropriate methods and were generally poorly reported. Using poor methods can affect the predictive ability of the models, whereas inadequate reporting hinders an objective evaluation of the potential usefulness of the model.

Original languageEnglish
Pages (from-to)268-277
Number of pages10
JournalJournal of Clinical Epidemiology
Volume66
Issue number3
DOIs
Publication statusPublished - 1 Mar 2013
Externally publishedYes

Fingerprint

Chronic Renal Insufficiency
Chronic Kidney Failure
Renal Insufficiency
Statistical Models
PubMed
Sample Size
Databases
Hypertension
Kidney
Incidence

Keywords

  • Kidney disease
  • Methodological conduct
  • Model development
  • Model validation
  • Reporting
  • Risk prediction models

ASJC Scopus subject areas

  • Epidemiology

Cite this

A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. / Collins, Gary S.; Omar, Omar; Shanyinde, Milensu; Yu, Ly Mee.

In: Journal of Clinical Epidemiology, Vol. 66, No. 3, 01.03.2013, p. 268-277.

Research output: Contribution to journalReview article

@article{6bea24f464aa40a48850286b52909bf8,
title = "A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods",
abstract = "Background: Chronic kidney disease (CKD) is a global health concern that is increasing mainly as the result of increasing incidences of diabetes and hypertension. Furthermore, if left untreated, individuals with CKD may progress to end-stage kidney failure. Identifying individuals with undiagnosed CKD or those who are at an increased risk of developing CKD or progressing to end-stage kidney disease (ESKD) is therefore an important challenge. We sought to systematically review and critically assess the conduct and reporting of methods used to develop risk prediction models for predicting the risk of having undiagnosed (prevalent) or future risk of developing (incident) CKD or end-stage kidney failure in adults. Methods: We conducted a systematic search of PubMed database to identify studies published up until September 2011 that describe the development of models combining two or more variables to predict the risk of prevalent or incident CKD or ESKD. We extracted key information that describes aspects of developing a prediction model, including the study design, data quality, sample size and number of events, outcome definition, risk predictor selection and coding, missing data, model-building strategies, and aspects of performance. Results: Eleven studies describing the development of 14 prediction models were included. Eight studies reported the development of 11 models to predict incident CKD or ESKD, whereas 3 studies developed models for prevalent CKD. A total of 97 candidate risk predictors were considered, and 43 different risk predictors featured in the 14 prediction models. A method, not recommended to select risk predictors for inclusion in the multivariate model, using statistical significance from univariate screening was carried out in six studies. Missing data were frequently poorly handled and reported with no mention of missing data in four studies; 4 studies explicitly excluded individuals with missing data, and only 2 studies used multiple imputation to replace missing values. Conclusion: We found that prediction models for chronic kidney were often developed using inappropriate methods and were generally poorly reported. Using poor methods can affect the predictive ability of the models, whereas inadequate reporting hinders an objective evaluation of the potential usefulness of the model.",
keywords = "Kidney disease, Methodological conduct, Model development, Model validation, Reporting, Risk prediction models",
author = "Collins, {Gary S.} and Omar Omar and Milensu Shanyinde and Yu, {Ly Mee}",
year = "2013",
month = "3",
day = "1",
doi = "10.1016/j.jclinepi.2012.06.020",
language = "English",
volume = "66",
pages = "268--277",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "3",

}

TY - JOUR

T1 - A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods

AU - Collins, Gary S.

AU - Omar, Omar

AU - Shanyinde, Milensu

AU - Yu, Ly Mee

PY - 2013/3/1

Y1 - 2013/3/1

N2 - Background: Chronic kidney disease (CKD) is a global health concern that is increasing mainly as the result of increasing incidences of diabetes and hypertension. Furthermore, if left untreated, individuals with CKD may progress to end-stage kidney failure. Identifying individuals with undiagnosed CKD or those who are at an increased risk of developing CKD or progressing to end-stage kidney disease (ESKD) is therefore an important challenge. We sought to systematically review and critically assess the conduct and reporting of methods used to develop risk prediction models for predicting the risk of having undiagnosed (prevalent) or future risk of developing (incident) CKD or end-stage kidney failure in adults. Methods: We conducted a systematic search of PubMed database to identify studies published up until September 2011 that describe the development of models combining two or more variables to predict the risk of prevalent or incident CKD or ESKD. We extracted key information that describes aspects of developing a prediction model, including the study design, data quality, sample size and number of events, outcome definition, risk predictor selection and coding, missing data, model-building strategies, and aspects of performance. Results: Eleven studies describing the development of 14 prediction models were included. Eight studies reported the development of 11 models to predict incident CKD or ESKD, whereas 3 studies developed models for prevalent CKD. A total of 97 candidate risk predictors were considered, and 43 different risk predictors featured in the 14 prediction models. A method, not recommended to select risk predictors for inclusion in the multivariate model, using statistical significance from univariate screening was carried out in six studies. Missing data were frequently poorly handled and reported with no mention of missing data in four studies; 4 studies explicitly excluded individuals with missing data, and only 2 studies used multiple imputation to replace missing values. Conclusion: We found that prediction models for chronic kidney were often developed using inappropriate methods and were generally poorly reported. Using poor methods can affect the predictive ability of the models, whereas inadequate reporting hinders an objective evaluation of the potential usefulness of the model.

AB - Background: Chronic kidney disease (CKD) is a global health concern that is increasing mainly as the result of increasing incidences of diabetes and hypertension. Furthermore, if left untreated, individuals with CKD may progress to end-stage kidney failure. Identifying individuals with undiagnosed CKD or those who are at an increased risk of developing CKD or progressing to end-stage kidney disease (ESKD) is therefore an important challenge. We sought to systematically review and critically assess the conduct and reporting of methods used to develop risk prediction models for predicting the risk of having undiagnosed (prevalent) or future risk of developing (incident) CKD or end-stage kidney failure in adults. Methods: We conducted a systematic search of PubMed database to identify studies published up until September 2011 that describe the development of models combining two or more variables to predict the risk of prevalent or incident CKD or ESKD. We extracted key information that describes aspects of developing a prediction model, including the study design, data quality, sample size and number of events, outcome definition, risk predictor selection and coding, missing data, model-building strategies, and aspects of performance. Results: Eleven studies describing the development of 14 prediction models were included. Eight studies reported the development of 11 models to predict incident CKD or ESKD, whereas 3 studies developed models for prevalent CKD. A total of 97 candidate risk predictors were considered, and 43 different risk predictors featured in the 14 prediction models. A method, not recommended to select risk predictors for inclusion in the multivariate model, using statistical significance from univariate screening was carried out in six studies. Missing data were frequently poorly handled and reported with no mention of missing data in four studies; 4 studies explicitly excluded individuals with missing data, and only 2 studies used multiple imputation to replace missing values. Conclusion: We found that prediction models for chronic kidney were often developed using inappropriate methods and were generally poorly reported. Using poor methods can affect the predictive ability of the models, whereas inadequate reporting hinders an objective evaluation of the potential usefulness of the model.

KW - Kidney disease

KW - Methodological conduct

KW - Model development

KW - Model validation

KW - Reporting

KW - Risk prediction models

UR - http://www.scopus.com/inward/record.url?scp=84872797683&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84872797683&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2012.06.020

DO - 10.1016/j.jclinepi.2012.06.020

M3 - Review article

VL - 66

SP - 268

EP - 277

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 3

ER -