Development of the biostatistics and clinical epidemiology skills (BACES) assessment for medical residents

Patrick B. Barlow, Gary Skolits, Robert Heidel, William Metheny, Tiffany Lee Smith

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background Although biostatistics and clinical epidemiology are essential for comprehending medical evidence, research has shown consistently low and variable knowledge among postgraduate medical trainees. Simultaneously, there has been an increase in the complexity of statistical methods among top-tier medical journals. Aims To develop the Biostatics and Clinical Epidemiology Skills (BACES) assessment by (1) establishing content validity evidence of the BACES; (2) examining the model fit of the BACES items to an Item Response Theory (IRT) model; and (3) comparing IRT item estimates with those of traditional Classical Test Theory (CTT) indices. Methods Thirty multiple choice questions were written to focus on interpreting clinical epidemiological and statistical methods. Content validity was assessed through a four-person expert review. The instrument was administered to 150 residents across three academic medical centres in southern USA during the autumn of 2013. Data were fit to a two-parameter logistic IRT model and the item difficulty, discrimination and examinee ability values were compared with traditional CTT item statistics. Results 147 assessments were used for analysis (mean (SD) score 14.38 (3.38)). Twenty-six items, 13 devoted to statistics and 13 to clinical epidemiology, successfully fit a two-parameter logistic IRT model. These estimates also significantly correlated with their comparable CTT values. Conclusions The strength of the BACES instrument was supported by (1) establishing content validity evidence; (2) fitting a sample of 147 residents’ responses to an IRT model; and (3) correlating the IRT estimates with their CTT values, which makes it a flexible yet rigorous instrument for measuring biostatistical and clinical epidemiological knowledge.

Original languageEnglish (US)
Pages (from-to)423-430
Number of pages8
JournalPostgraduate Medical Journal
Volume91
Issue number1078
DOIs
StatePublished - Jan 8 2015

Fingerprint

Biostatistics
Clinical Competence
Epidemiology
Epidemiologic Methods
Biomedical Research

All Science Journal Classification (ASJC) codes

  • Medicine(all)

Cite this

Development of the biostatistics and clinical epidemiology skills (BACES) assessment for medical residents. / Barlow, Patrick B.; Skolits, Gary; Heidel, Robert; Metheny, William; Smith, Tiffany Lee.

In: Postgraduate Medical Journal, Vol. 91, No. 1078, 08.01.2015, p. 423-430.

Research output: Contribution to journalArticle

@article{796c992f2e574ce5a85985d81ee053c9,
title = "Development of the biostatistics and clinical epidemiology skills (BACES) assessment for medical residents",
abstract = "Background Although biostatistics and clinical epidemiology are essential for comprehending medical evidence, research has shown consistently low and variable knowledge among postgraduate medical trainees. Simultaneously, there has been an increase in the complexity of statistical methods among top-tier medical journals. Aims To develop the Biostatics and Clinical Epidemiology Skills (BACES) assessment by (1) establishing content validity evidence of the BACES; (2) examining the model fit of the BACES items to an Item Response Theory (IRT) model; and (3) comparing IRT item estimates with those of traditional Classical Test Theory (CTT) indices. Methods Thirty multiple choice questions were written to focus on interpreting clinical epidemiological and statistical methods. Content validity was assessed through a four-person expert review. The instrument was administered to 150 residents across three academic medical centres in southern USA during the autumn of 2013. Data were fit to a two-parameter logistic IRT model and the item difficulty, discrimination and examinee ability values were compared with traditional CTT item statistics. Results 147 assessments were used for analysis (mean (SD) score 14.38 (3.38)). Twenty-six items, 13 devoted to statistics and 13 to clinical epidemiology, successfully fit a two-parameter logistic IRT model. These estimates also significantly correlated with their comparable CTT values. Conclusions The strength of the BACES instrument was supported by (1) establishing content validity evidence; (2) fitting a sample of 147 residents’ responses to an IRT model; and (3) correlating the IRT estimates with their CTT values, which makes it a flexible yet rigorous instrument for measuring biostatistical and clinical epidemiological knowledge.",
author = "Barlow, {Patrick B.} and Gary Skolits and Robert Heidel and William Metheny and Smith, {Tiffany Lee}",
year = "2015",
month = "1",
day = "8",
doi = "10.1136/postgradmedj-2014-133197",
language = "English (US)",
volume = "91",
pages = "423--430",
journal = "Postgraduate Medical Journal",
issn = "0032-5473",
publisher = "BMJ Publishing Group",
number = "1078",

}

TY - JOUR

T1 - Development of the biostatistics and clinical epidemiology skills (BACES) assessment for medical residents

AU - Barlow, Patrick B.

AU - Skolits, Gary

AU - Heidel, Robert

AU - Metheny, William

AU - Smith, Tiffany Lee

PY - 2015/1/8

Y1 - 2015/1/8

N2 - Background Although biostatistics and clinical epidemiology are essential for comprehending medical evidence, research has shown consistently low and variable knowledge among postgraduate medical trainees. Simultaneously, there has been an increase in the complexity of statistical methods among top-tier medical journals. Aims To develop the Biostatics and Clinical Epidemiology Skills (BACES) assessment by (1) establishing content validity evidence of the BACES; (2) examining the model fit of the BACES items to an Item Response Theory (IRT) model; and (3) comparing IRT item estimates with those of traditional Classical Test Theory (CTT) indices. Methods Thirty multiple choice questions were written to focus on interpreting clinical epidemiological and statistical methods. Content validity was assessed through a four-person expert review. The instrument was administered to 150 residents across three academic medical centres in southern USA during the autumn of 2013. Data were fit to a two-parameter logistic IRT model and the item difficulty, discrimination and examinee ability values were compared with traditional CTT item statistics. Results 147 assessments were used for analysis (mean (SD) score 14.38 (3.38)). Twenty-six items, 13 devoted to statistics and 13 to clinical epidemiology, successfully fit a two-parameter logistic IRT model. These estimates also significantly correlated with their comparable CTT values. Conclusions The strength of the BACES instrument was supported by (1) establishing content validity evidence; (2) fitting a sample of 147 residents’ responses to an IRT model; and (3) correlating the IRT estimates with their CTT values, which makes it a flexible yet rigorous instrument for measuring biostatistical and clinical epidemiological knowledge.

AB - Background Although biostatistics and clinical epidemiology are essential for comprehending medical evidence, research has shown consistently low and variable knowledge among postgraduate medical trainees. Simultaneously, there has been an increase in the complexity of statistical methods among top-tier medical journals. Aims To develop the Biostatics and Clinical Epidemiology Skills (BACES) assessment by (1) establishing content validity evidence of the BACES; (2) examining the model fit of the BACES items to an Item Response Theory (IRT) model; and (3) comparing IRT item estimates with those of traditional Classical Test Theory (CTT) indices. Methods Thirty multiple choice questions were written to focus on interpreting clinical epidemiological and statistical methods. Content validity was assessed through a four-person expert review. The instrument was administered to 150 residents across three academic medical centres in southern USA during the autumn of 2013. Data were fit to a two-parameter logistic IRT model and the item difficulty, discrimination and examinee ability values were compared with traditional CTT item statistics. Results 147 assessments were used for analysis (mean (SD) score 14.38 (3.38)). Twenty-six items, 13 devoted to statistics and 13 to clinical epidemiology, successfully fit a two-parameter logistic IRT model. These estimates also significantly correlated with their comparable CTT values. Conclusions The strength of the BACES instrument was supported by (1) establishing content validity evidence; (2) fitting a sample of 147 residents’ responses to an IRT model; and (3) correlating the IRT estimates with their CTT values, which makes it a flexible yet rigorous instrument for measuring biostatistical and clinical epidemiological knowledge.

UR - http://www.scopus.com/inward/record.url?scp=84940453447&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940453447&partnerID=8YFLogxK

U2 - 10.1136/postgradmedj-2014-133197

DO - 10.1136/postgradmedj-2014-133197

M3 - Article

VL - 91

SP - 423

EP - 430

JO - Postgraduate Medical Journal

JF - Postgraduate Medical Journal

SN - 0032-5473

IS - 1078

ER -