- Deep learning and natural language processing techniques can significantly improve the detection of adverse drug events (ADEs) in unstructured electronic health record (EHR) data, a study published in JMIR Medical Informatics revealed.
Researchers developed a deep learning model that demonstrated an accuracy of 65.9 percent, surpassing an existing model that performs with an accuracy of just 61.7 percent.
These results are significant for organizations seeking to reduce care costs and improve patient safety. ADEs account for 41 percent of all hospital admissions, the team noted, and these events can result in significantly higher medical expenses and poor outcomes for patients.
The researchers said that other investigators have previously used EHR notes as a resource to identify ADEs, but manually extracting information from this data can be tedious. While NLP techniques have emerged as a viable way to detect ADEs in unstructured data, the team pointed out that few studies have examined the effects of using a deep learning approach along with NLP to flag these events.
To evaluate the accuracy of a deep learning technique, researchers developed a model consisting of two sub-networks: one designed to recognize specific words, such as “medication,” “frequency,” “severity,” and “dosage”; and another designed to recognize predefined relations between these words, such as dosage-medication and frequency-medication.
The team tested the model on EHR data, and then compared its performance to an existing model. The results showed that the deep learning model performed with 4.2 percent greater overall accuracy than the existing model.
The deep learning model significantly outperformed the existing system when detecting relations between entities. The model achieved 81 percent accuracy when identifying dosage-medication relations, while the existing system identified these relations with an accuracy of just 6.2 percent.
Additionally, the deep learning approach identified dosage-frequency relations within the data with an accuracy of 78.4 percent, while the existing system identified these relations with 42.8 percent accuracy. These results indicate that deep learning methods may play a key role in improving patient safety.
These findings add to previous research demonstrating the ability of NLP and machine learning techniques to identify ADEs in unstructured data. A 2017 study showed that these innovative techniques significantly outperformed traditional detection methods when applied to medical literature and social media postings.
While the model developed in the current study performed with greater accuracy than an existing system, the deep learning approach did make some errors.
The team noted that the model was most likely to make false-negative errors when it came across long expressions consisting of multiple words. Additionally, the model missed relations between two words when they were more than six sentences apart from each other in the EHR.
These results are also consistent with recent research. While deep learning algorithms have shown success in multiple areas of care delivery, an October 2018 study found that researchers still face unique challenges when training these tools on EHR data, including unstructured clinical notes.
“Results from the reviewed articles have shown that as compared to other machine learning approaches, deep learning models excel in modeling raw data, minimizing the need for preprocessing and feature engineering, and significantly improving performance in many analytical tasks,” the study said.
“However, although deep learning techniques have shown promising results in performing many analytics tasks, several open challenges remain.”
Although deep learning tools still have some way to go, the results of the current study show that this technology has the potential to improve the detection of adverse patient safety events.
“Our model achieved state-of-the-art performance in an ADE-detection dataset,” the researchers said. “The results show that deep learning models can significantly improve the performance of ADE-related information extraction.”