Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis

Christine Y. Lu, Robert B. Penfold, Sengwee Toh, Jessica L. Sturtevant, Jeanne M. Madden, Gregory Simon, Brian K. Ahmedani, Gregory Clarke, Karen J. Coleman, Laurel A. Copeland, Yihe G. Daida, Robert Davis, Enid M. Hunkeler, Ashli Owen-Smith, Marsha A. Raebel, Rebecca Rossom, Stephen B. Soumerai, Martin Kulldorff

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Background: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. Objective: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. Method: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. Results: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. Conclusions: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.

Original languageEnglish (US)
Pages (from-to)365-372
Number of pages8
JournalMedical care
Volume56
Issue number5
DOIs
StatePublished - Jan 1 2018

Fingerprint

Health Policy
Suicide
Psychotropic Drugs
Poisoning
Young Adult
Population Surveillance
Proxy
United States Food and Drug Administration
Risk-Taking
Antidepressive Agents
Longitudinal Studies
Mental Health
Cohort Studies
Retrospective Studies
Age Groups
Outcome Assessment (Health Care)
Delivery of Health Care
Health
Research
Population

All Science Journal Classification (ASJC) codes

  • Public Health, Environmental and Occupational Health

Cite this

Lu, C. Y., Penfold, R. B., Toh, S., Sturtevant, J. L., Madden, J. M., Simon, G., ... Kulldorff, M. (2018). Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis. Medical care, 56(5), 365-372. https://doi.org/10.1097/MLR.0000000000000893

Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis. / Lu, Christine Y.; Penfold, Robert B.; Toh, Sengwee; Sturtevant, Jessica L.; Madden, Jeanne M.; Simon, Gregory; Ahmedani, Brian K.; Clarke, Gregory; Coleman, Karen J.; Copeland, Laurel A.; Daida, Yihe G.; Davis, Robert; Hunkeler, Enid M.; Owen-Smith, Ashli; Raebel, Marsha A.; Rossom, Rebecca; Soumerai, Stephen B.; Kulldorff, Martin.

In: Medical care, Vol. 56, No. 5, 01.01.2018, p. 365-372.

Research output: Contribution to journalArticle

Lu, CY, Penfold, RB, Toh, S, Sturtevant, JL, Madden, JM, Simon, G, Ahmedani, BK, Clarke, G, Coleman, KJ, Copeland, LA, Daida, YG, Davis, R, Hunkeler, EM, Owen-Smith, A, Raebel, MA, Rossom, R, Soumerai, SB & Kulldorff, M 2018, 'Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis', Medical care, vol. 56, no. 5, pp. 365-372. https://doi.org/10.1097/MLR.0000000000000893
Lu CY, Penfold RB, Toh S, Sturtevant JL, Madden JM, Simon G et al. Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis. Medical care. 2018 Jan 1;56(5):365-372. https://doi.org/10.1097/MLR.0000000000000893
Lu, Christine Y. ; Penfold, Robert B. ; Toh, Sengwee ; Sturtevant, Jessica L. ; Madden, Jeanne M. ; Simon, Gregory ; Ahmedani, Brian K. ; Clarke, Gregory ; Coleman, Karen J. ; Copeland, Laurel A. ; Daida, Yihe G. ; Davis, Robert ; Hunkeler, Enid M. ; Owen-Smith, Ashli ; Raebel, Marsha A. ; Rossom, Rebecca ; Soumerai, Stephen B. ; Kulldorff, Martin. / Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis. In: Medical care. 2018 ; Vol. 56, No. 5. pp. 365-372.
@article{02f1cbdaa6ba483cb672f9b8475ed9cf,
title = "Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis",
abstract = "Background: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. Objective: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. Method: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. Results: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. Conclusions: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.",
author = "Lu, {Christine Y.} and Penfold, {Robert B.} and Sengwee Toh and Sturtevant, {Jessica L.} and Madden, {Jeanne M.} and Gregory Simon and Ahmedani, {Brian K.} and Gregory Clarke and Coleman, {Karen J.} and Copeland, {Laurel A.} and Daida, {Yihe G.} and Robert Davis and Hunkeler, {Enid M.} and Ashli Owen-Smith and Raebel, {Marsha A.} and Rebecca Rossom and Soumerai, {Stephen B.} and Martin Kulldorff",
year = "2018",
month = "1",
day = "1",
doi = "10.1097/MLR.0000000000000893",
language = "English (US)",
volume = "56",
pages = "365--372",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "5",

}

TY - JOUR

T1 - Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis

AU - Lu, Christine Y.

AU - Penfold, Robert B.

AU - Toh, Sengwee

AU - Sturtevant, Jessica L.

AU - Madden, Jeanne M.

AU - Simon, Gregory

AU - Ahmedani, Brian K.

AU - Clarke, Gregory

AU - Coleman, Karen J.

AU - Copeland, Laurel A.

AU - Daida, Yihe G.

AU - Davis, Robert

AU - Hunkeler, Enid M.

AU - Owen-Smith, Ashli

AU - Raebel, Marsha A.

AU - Rossom, Rebecca

AU - Soumerai, Stephen B.

AU - Kulldorff, Martin

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Background: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. Objective: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. Method: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. Results: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. Conclusions: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.

AB - Background: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. Objective: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. Method: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. Results: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. Conclusions: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.

UR - http://www.scopus.com/inward/record.url?scp=85045633216&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045633216&partnerID=8YFLogxK

U2 - 10.1097/MLR.0000000000000893

DO - 10.1097/MLR.0000000000000893

M3 - Article

C2 - 29634627

AN - SCOPUS:85045633216

VL - 56

SP - 365

EP - 372

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 5

ER -