Activity ID
13591Expires
August 16, 2027Format Type
Journal-basedCME Credit
1Fee
$30CME Provider: JAMA Network Open
Description of CME Course
Importance The Sentinel System is a key component of the US Food and Drug Administration (FDA) postmarketing safety surveillance commitment and uses clinical health care data to conduct analyses to inform drug labeling and safety communications, FDA advisory committee meetings, and other regulatory decisions. However, observational data are frequently deemed insufficient for reliable evaluation of safety concerns owing to limitations in underlying data or methodology. Advances in large language models (LLMs) provide new opportunities to address some of these limitations. However, careful consideration is necessary for how and where LLMs can be effectively deployed for these purposes.
Observations LLMs may provide new avenues to support signal-identification activities to identify novel adverse event signals from narrative text of electronic health records. These algorithms may be used to support epidemiologic investigations examining the causal relationship between exposure to a medical product and an adverse event through development of probabilistic phenotyping of health outcomes of interest and extraction of information related to important confounding factors. LLMs may perform like traditional natural language processing tools by annotating text with controlled vocabularies with additional tailored training activities. LLMs offer opportunities for enhancing information extraction from adverse event reports, medical literature, and other biomedical knowledge sources. There are several challenges that must be considered when leveraging LLMs for postmarket surveillance. Prompt engineering is needed to ensure that LLM-extracted associations are accurate and specific. LLMs require extensive infrastructure to use, which many health care systems lack, and this can impact diversity, equity, and inclusion, and result in obscuring significant adverse event patterns in some populations. LLMs are known to generate nonfactual statements, which could lead to false positive signals and downstream evaluation activities by the FDA and other entities, incurring substantial cost.
Conclusions and Relevance LLMs represent a novel paradigm that may facilitate generation of information to support medical product postmarket surveillance activities that have not been possible. However, additional work is required to ensure LLMs can be used in a fair and equitable manner, minimize false positive findings, and support the necessary rigor of signal detection needed for regulatory activities.
Disclaimers
1. This activity is accredited by the American Medical Association.
2. This activity is free to AMA members.
ABMS Member Board Approvals by Type
ABMS Lifelong Learning CME Activity
Allergy and Immunology
Anesthesiology
Colon and Rectal Surgery
Family Medicine
Medical Genetics and Genomics
Nuclear Medicine
Ophthalmology
Pathology
Physical Medicine and Rehabilitation
Plastic Surgery
Preventive Medicine
Psychiatry and Neurology
Radiology
Thoracic Surgery
Urology
Commercial Support?
NoNOTE: If a Member Board has not deemed this activity for MOC approval as an accredited CME activity, this activity may count toward an ABMS Member Board’s general CME requirement. Please refer directly to your Member Board’s MOC Part II Lifelong Learning and Self-Assessment Program Requirements.
Educational Objectives
To identify the key insights or developments described in this article
Keywords
Adverse Drug Events, Clinical Pharmacy and Pharmacology, Health Care Safety, Health Policy, Pharmacy and Clinical Pharmacology
Competencies
Medical Knowledge
CME Credit Type
AMA PRA Category 1 Credit
DOI
10.1001/jamanetworkopen.2024.28276