Validation study of artificial neural network models for prediction of methicillin-resistant Staphylococcus aureus carriage

Cheng Chuan Hsu, Yusen E. Lin, Yao Shen Chen, Yung Ching Liu, Robert R. Muder

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

OBJECTIVE. Use of active surveillance cultures for methicillin-resistant Staphylococcus aureus (MRSA) for all patients admitted to the intensive care unit has been shown to reduce nosocomial transmission. However, the cost-effectiveness and the utility of implementing use of active surveillance cultures nationwide remain controversial. We sought to develop an artificial neural network (ANN) model that would predict the likelihood of MRSA colonization. SETTING. Two acute care hospitals, one in Pittsburgh (hospital A) and one in Kaohsiung, Taiwan (hospital B). METHODS. Nasal cultures were performed for all patients admitted to the hospitals. A total of 46 potential risk factors in hospital A and 86 potential risk factors in hospital B associated with MRSA colonization were assessed. Culture results were obtained; 75% of the data were used for training our ANN model, and the remaining 25% were used for validating our ANN model. The culture results were the "gold standard" for determining the accuracy of the model predictions. RESULTS. The ANN model predictions were accurate 95.2% of the time for hospital A (sensitivity, 94.3%; specificity, 96.0%) and 94.2% of the time for hospital B (sensitivity, 96.6%; specificity, 91.8%), integrating all potential risk factors into the model. Only 17 potential risk factors were needed for the hospital AANN model (accuracy, 90.9%; sensitivity, 98.5%; specificity, 83.4%), and only 20 potential risk factors were needed for the hospital BANN model (accuracy, 90.5%; sensitivity, 96.6%; specificity, 84.3%), if the minimal risk factor method was used. Cross-validation analysis showed an average accuracy of 85.6% (sensitivity, 91.3%; specificity, 80.0%). CONCLUSION. Our ANN model can be used to predict with an accuracy of more than 90% which patients carry MRSA. The false-negative rates were significantly lower than the false-positive rates in the ANN predictions, which can serve as a safety buffer in case of patient misclassification.

Original languageEnglish
Pages (from-to)607-614
Number of pages8
JournalInfection Control and Hospital Epidemiology
Volume29
Issue number7
DOIs
Publication statusPublished - Jul 2008
Externally publishedYes

Fingerprint

Neural Networks (Computer)
Validation Studies
Methicillin-Resistant Staphylococcus aureus
Taiwan
Nose
Cost-Benefit Analysis
Intensive Care Units
Buffers

ASJC Scopus subject areas

  • Immunology
  • Microbiology (medical)

Cite this

Validation study of artificial neural network models for prediction of methicillin-resistant Staphylococcus aureus carriage. / Hsu, Cheng Chuan; Lin, Yusen E.; Chen, Yao Shen; Liu, Yung Ching; Muder, Robert R.

In: Infection Control and Hospital Epidemiology, Vol. 29, No. 7, 07.2008, p. 607-614.

Research output: Contribution to journalArticle

@article{761478f7af1a487a8623933a764ad2d3,
title = "Validation study of artificial neural network models for prediction of methicillin-resistant Staphylococcus aureus carriage",
abstract = "OBJECTIVE. Use of active surveillance cultures for methicillin-resistant Staphylococcus aureus (MRSA) for all patients admitted to the intensive care unit has been shown to reduce nosocomial transmission. However, the cost-effectiveness and the utility of implementing use of active surveillance cultures nationwide remain controversial. We sought to develop an artificial neural network (ANN) model that would predict the likelihood of MRSA colonization. SETTING. Two acute care hospitals, one in Pittsburgh (hospital A) and one in Kaohsiung, Taiwan (hospital B). METHODS. Nasal cultures were performed for all patients admitted to the hospitals. A total of 46 potential risk factors in hospital A and 86 potential risk factors in hospital B associated with MRSA colonization were assessed. Culture results were obtained; 75{\%} of the data were used for training our ANN model, and the remaining 25{\%} were used for validating our ANN model. The culture results were the {"}gold standard{"} for determining the accuracy of the model predictions. RESULTS. The ANN model predictions were accurate 95.2{\%} of the time for hospital A (sensitivity, 94.3{\%}; specificity, 96.0{\%}) and 94.2{\%} of the time for hospital B (sensitivity, 96.6{\%}; specificity, 91.8{\%}), integrating all potential risk factors into the model. Only 17 potential risk factors were needed for the hospital AANN model (accuracy, 90.9{\%}; sensitivity, 98.5{\%}; specificity, 83.4{\%}), and only 20 potential risk factors were needed for the hospital BANN model (accuracy, 90.5{\%}; sensitivity, 96.6{\%}; specificity, 84.3{\%}), if the minimal risk factor method was used. Cross-validation analysis showed an average accuracy of 85.6{\%} (sensitivity, 91.3{\%}; specificity, 80.0{\%}). CONCLUSION. Our ANN model can be used to predict with an accuracy of more than 90{\%} which patients carry MRSA. The false-negative rates were significantly lower than the false-positive rates in the ANN predictions, which can serve as a safety buffer in case of patient misclassification.",
author = "Hsu, {Cheng Chuan} and Lin, {Yusen E.} and Chen, {Yao Shen} and Liu, {Yung Ching} and Muder, {Robert R.}",
year = "2008",
month = "7",
doi = "10.1086/588588",
language = "English",
volume = "29",
pages = "607--614",
journal = "Infection Control and Hospital Epidemiology",
issn = "0899-823X",
publisher = "University of Chicago Press",
number = "7",

}

TY - JOUR

T1 - Validation study of artificial neural network models for prediction of methicillin-resistant Staphylococcus aureus carriage

AU - Hsu, Cheng Chuan

AU - Lin, Yusen E.

AU - Chen, Yao Shen

AU - Liu, Yung Ching

AU - Muder, Robert R.

PY - 2008/7

Y1 - 2008/7

N2 - OBJECTIVE. Use of active surveillance cultures for methicillin-resistant Staphylococcus aureus (MRSA) for all patients admitted to the intensive care unit has been shown to reduce nosocomial transmission. However, the cost-effectiveness and the utility of implementing use of active surveillance cultures nationwide remain controversial. We sought to develop an artificial neural network (ANN) model that would predict the likelihood of MRSA colonization. SETTING. Two acute care hospitals, one in Pittsburgh (hospital A) and one in Kaohsiung, Taiwan (hospital B). METHODS. Nasal cultures were performed for all patients admitted to the hospitals. A total of 46 potential risk factors in hospital A and 86 potential risk factors in hospital B associated with MRSA colonization were assessed. Culture results were obtained; 75% of the data were used for training our ANN model, and the remaining 25% were used for validating our ANN model. The culture results were the "gold standard" for determining the accuracy of the model predictions. RESULTS. The ANN model predictions were accurate 95.2% of the time for hospital A (sensitivity, 94.3%; specificity, 96.0%) and 94.2% of the time for hospital B (sensitivity, 96.6%; specificity, 91.8%), integrating all potential risk factors into the model. Only 17 potential risk factors were needed for the hospital AANN model (accuracy, 90.9%; sensitivity, 98.5%; specificity, 83.4%), and only 20 potential risk factors were needed for the hospital BANN model (accuracy, 90.5%; sensitivity, 96.6%; specificity, 84.3%), if the minimal risk factor method was used. Cross-validation analysis showed an average accuracy of 85.6% (sensitivity, 91.3%; specificity, 80.0%). CONCLUSION. Our ANN model can be used to predict with an accuracy of more than 90% which patients carry MRSA. The false-negative rates were significantly lower than the false-positive rates in the ANN predictions, which can serve as a safety buffer in case of patient misclassification.

AB - OBJECTIVE. Use of active surveillance cultures for methicillin-resistant Staphylococcus aureus (MRSA) for all patients admitted to the intensive care unit has been shown to reduce nosocomial transmission. However, the cost-effectiveness and the utility of implementing use of active surveillance cultures nationwide remain controversial. We sought to develop an artificial neural network (ANN) model that would predict the likelihood of MRSA colonization. SETTING. Two acute care hospitals, one in Pittsburgh (hospital A) and one in Kaohsiung, Taiwan (hospital B). METHODS. Nasal cultures were performed for all patients admitted to the hospitals. A total of 46 potential risk factors in hospital A and 86 potential risk factors in hospital B associated with MRSA colonization were assessed. Culture results were obtained; 75% of the data were used for training our ANN model, and the remaining 25% were used for validating our ANN model. The culture results were the "gold standard" for determining the accuracy of the model predictions. RESULTS. The ANN model predictions were accurate 95.2% of the time for hospital A (sensitivity, 94.3%; specificity, 96.0%) and 94.2% of the time for hospital B (sensitivity, 96.6%; specificity, 91.8%), integrating all potential risk factors into the model. Only 17 potential risk factors were needed for the hospital AANN model (accuracy, 90.9%; sensitivity, 98.5%; specificity, 83.4%), and only 20 potential risk factors were needed for the hospital BANN model (accuracy, 90.5%; sensitivity, 96.6%; specificity, 84.3%), if the minimal risk factor method was used. Cross-validation analysis showed an average accuracy of 85.6% (sensitivity, 91.3%; specificity, 80.0%). CONCLUSION. Our ANN model can be used to predict with an accuracy of more than 90% which patients carry MRSA. The false-negative rates were significantly lower than the false-positive rates in the ANN predictions, which can serve as a safety buffer in case of patient misclassification.

UR - http://www.scopus.com/inward/record.url?scp=47549119492&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=47549119492&partnerID=8YFLogxK

U2 - 10.1086/588588

DO - 10.1086/588588

M3 - Article

VL - 29

SP - 607

EP - 614

JO - Infection Control and Hospital Epidemiology

JF - Infection Control and Hospital Epidemiology

SN - 0899-823X

IS - 7

ER -