Receiver operating characteristic curve-based prediction model for periodontal disease updated with the calibrated community periodontal index

Chiu Wen Su, Amy Ming Fang Yen, Hongmin Lai, Hsiu Hsi Chen, Sam Li Sheng Chen

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: The accuracy of a prediction model for periodontal disease using the community periodontal index (CPI) has been undertaken by using an area under a receiver operating characteristics (AUROC) curve. How the uncalibrated CPI, as measured by general dentists trained by periodontists in a large epidemiologic study, and affects the performance in a prediction model, has not been researched yet. Methods: A two-stage design was conducted by first proposing a validation study to calibrate CPI between a senior periodontal specialist and trained general dentists who measured CPIs in the main study of a nationwide survey. A Bayesian hierarchical logistic regression model was applied to estimate the non-updated and updated clinical weights used for building up risk scores. How the calibrated CPI affected performance of the updated prediction model was quantified by comparing AUROC curves between the original and updated models. Results: Estimates regarding calibration of CPI obtained from the validation study were 66% and 85% for sensitivity and specificity, respectively. After updating, clinical weights of each predictor were inflated, and the risk score for the highest risk category was elevated from 434 to 630. Such an update improved the AUROC performance of the two corresponding prediction models from 62.6% (95% confidence interval [CI]: 61.7% to 63.6%) for the non-updated model to 68.9% (95% CI: 68.0% to 69.6%) for the updated one, reaching a statistically significant difference (P <0.05). Conclusion: An improvement in the updated prediction model was demonstrated for periodontal disease as measured by the calibrated CPI derived from a large epidemiologic survey.

Original languageEnglish
Pages (from-to)1348-1355
Number of pages8
JournalJournal of Periodontology
Volume88
Issue number12
DOIs
Publication statusPublished - Dec 1 2017

Fingerprint

Periodontal Index
Periodontal Diseases
ROC Curve
Dentists
Validation Studies
Logistic Models
Confidence Intervals
Weights and Measures
Calibration
Epidemiologic Studies
Sensitivity and Specificity

Keywords

  • Periodontal diseases
  • Periodontal index
  • ROC curve

ASJC Scopus subject areas

  • Periodontics

Cite this

Receiver operating characteristic curve-based prediction model for periodontal disease updated with the calibrated community periodontal index. / Su, Chiu Wen; Yen, Amy Ming Fang; Lai, Hongmin; Chen, Hsiu Hsi; Chen, Sam Li Sheng.

In: Journal of Periodontology, Vol. 88, No. 12, 01.12.2017, p. 1348-1355.

Research output: Contribution to journalArticle

@article{438701df9219460693324521b2be7a93,
title = "Receiver operating characteristic curve-based prediction model for periodontal disease updated with the calibrated community periodontal index",
abstract = "Background: The accuracy of a prediction model for periodontal disease using the community periodontal index (CPI) has been undertaken by using an area under a receiver operating characteristics (AUROC) curve. How the uncalibrated CPI, as measured by general dentists trained by periodontists in a large epidemiologic study, and affects the performance in a prediction model, has not been researched yet. Methods: A two-stage design was conducted by first proposing a validation study to calibrate CPI between a senior periodontal specialist and trained general dentists who measured CPIs in the main study of a nationwide survey. A Bayesian hierarchical logistic regression model was applied to estimate the non-updated and updated clinical weights used for building up risk scores. How the calibrated CPI affected performance of the updated prediction model was quantified by comparing AUROC curves between the original and updated models. Results: Estimates regarding calibration of CPI obtained from the validation study were 66{\%} and 85{\%} for sensitivity and specificity, respectively. After updating, clinical weights of each predictor were inflated, and the risk score for the highest risk category was elevated from 434 to 630. Such an update improved the AUROC performance of the two corresponding prediction models from 62.6{\%} (95{\%} confidence interval [CI]: 61.7{\%} to 63.6{\%}) for the non-updated model to 68.9{\%} (95{\%} CI: 68.0{\%} to 69.6{\%}) for the updated one, reaching a statistically significant difference (P <0.05). Conclusion: An improvement in the updated prediction model was demonstrated for periodontal disease as measured by the calibrated CPI derived from a large epidemiologic survey.",
keywords = "Periodontal diseases, Periodontal index, ROC curve",
author = "Su, {Chiu Wen} and Yen, {Amy Ming Fang} and Hongmin Lai and Chen, {Hsiu Hsi} and Chen, {Sam Li Sheng}",
year = "2017",
month = "12",
day = "1",
doi = "10.1902/jop.2017.170138",
language = "English",
volume = "88",
pages = "1348--1355",
journal = "Journal of Periodontology",
issn = "0022-3492",
publisher = "American Academy of Periodontology",
number = "12",

}

TY - JOUR

T1 - Receiver operating characteristic curve-based prediction model for periodontal disease updated with the calibrated community periodontal index

AU - Su, Chiu Wen

AU - Yen, Amy Ming Fang

AU - Lai, Hongmin

AU - Chen, Hsiu Hsi

AU - Chen, Sam Li Sheng

PY - 2017/12/1

Y1 - 2017/12/1

N2 - Background: The accuracy of a prediction model for periodontal disease using the community periodontal index (CPI) has been undertaken by using an area under a receiver operating characteristics (AUROC) curve. How the uncalibrated CPI, as measured by general dentists trained by periodontists in a large epidemiologic study, and affects the performance in a prediction model, has not been researched yet. Methods: A two-stage design was conducted by first proposing a validation study to calibrate CPI between a senior periodontal specialist and trained general dentists who measured CPIs in the main study of a nationwide survey. A Bayesian hierarchical logistic regression model was applied to estimate the non-updated and updated clinical weights used for building up risk scores. How the calibrated CPI affected performance of the updated prediction model was quantified by comparing AUROC curves between the original and updated models. Results: Estimates regarding calibration of CPI obtained from the validation study were 66% and 85% for sensitivity and specificity, respectively. After updating, clinical weights of each predictor were inflated, and the risk score for the highest risk category was elevated from 434 to 630. Such an update improved the AUROC performance of the two corresponding prediction models from 62.6% (95% confidence interval [CI]: 61.7% to 63.6%) for the non-updated model to 68.9% (95% CI: 68.0% to 69.6%) for the updated one, reaching a statistically significant difference (P <0.05). Conclusion: An improvement in the updated prediction model was demonstrated for periodontal disease as measured by the calibrated CPI derived from a large epidemiologic survey.

AB - Background: The accuracy of a prediction model for periodontal disease using the community periodontal index (CPI) has been undertaken by using an area under a receiver operating characteristics (AUROC) curve. How the uncalibrated CPI, as measured by general dentists trained by periodontists in a large epidemiologic study, and affects the performance in a prediction model, has not been researched yet. Methods: A two-stage design was conducted by first proposing a validation study to calibrate CPI between a senior periodontal specialist and trained general dentists who measured CPIs in the main study of a nationwide survey. A Bayesian hierarchical logistic regression model was applied to estimate the non-updated and updated clinical weights used for building up risk scores. How the calibrated CPI affected performance of the updated prediction model was quantified by comparing AUROC curves between the original and updated models. Results: Estimates regarding calibration of CPI obtained from the validation study were 66% and 85% for sensitivity and specificity, respectively. After updating, clinical weights of each predictor were inflated, and the risk score for the highest risk category was elevated from 434 to 630. Such an update improved the AUROC performance of the two corresponding prediction models from 62.6% (95% confidence interval [CI]: 61.7% to 63.6%) for the non-updated model to 68.9% (95% CI: 68.0% to 69.6%) for the updated one, reaching a statistically significant difference (P <0.05). Conclusion: An improvement in the updated prediction model was demonstrated for periodontal disease as measured by the calibrated CPI derived from a large epidemiologic survey.

KW - Periodontal diseases

KW - Periodontal index

KW - ROC curve

UR - http://www.scopus.com/inward/record.url?scp=85035344537&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035344537&partnerID=8YFLogxK

U2 - 10.1902/jop.2017.170138

DO - 10.1902/jop.2017.170138

M3 - Article

VL - 88

SP - 1348

EP - 1355

JO - Journal of Periodontology

JF - Journal of Periodontology

SN - 0022-3492

IS - 12

ER -