Natural Language Processing Advances Clinical Decision Making

December 27, 2019 - Providers are inundated with large amounts of data that is daunting to comb through to find actionable outcomes. 

Artificial intelligence (AI) technology is frequently seen as the solution to this problem, as it boasts faster and more accurate interpretation time than most providers. 

Yet providers are hesitant to integrate this technology into clinical workflows. Black box AI tools make many providers weary about how the machine is making its decisions, as well as the accuracy of these outputs. Recent studies showing racial bias perpetuated by the use of this technology also contributes to providers hesitancy to adopt the technology. 

Machine learning and natural language processing that is developed within an organization rather than relying on outside vendors can improve providers’ confidence in integrating AI into their workflows. 

Kelly Denney, director of data science strategy at Radiology Partners, spoke with HealthITAnalytics.com about the AI algorithms they are developing on-site to improve radiology workflow.  

“We are using machine learning and natural language processing to have a real-time listener operating at the same time that our radiologists are dictating the report. It recognizes when certain conditions are described in the radiology report,” Denney said. “We’re able to provide the best practice recommendation to the radiologist so that they can implement standardized care within their reports every time. It’s really a workflow tool.” 

Instead of being housed under the umbrella of IT, Denney’s department is under clinical value. This allows their data science initiatives to focus specifically on improving the clinical workflow and outcomes for their patients.  

“We’re not strictly in the technology space,” Denney explained. “Anything that improves patient outcomes and decreases costs for providers and payers is in the realm under which we work.” 

Developing the technology internally has helped Radiology Partners open up the black box. 

“Radiologists can, if they’re interested and not quite sure, can go in and see what our models found in the report to lead to the recommendations that we provided,” Denney revealed. 

“We have multiple feedback mechanisms so that radiologists using the tool in real time can notify the Data Science Team if there are any false positives or false negatives,” she continued. “That real time interactive capability within the user interface of the tool comes directly back to the Data Science Team so that we can look up that case.” 

Each month, the team pulls cases from the data warehouse, runs them through the algorithm, and compares the results to the actual provider recommendation. 

“We have human receives flag cases that we didn’t correctly act upon,” Denney noted. “We retrain the models based on that feedback.”

Monitoring these errors allows Denney’s team to understand common mistakes made by the algorithm and correct them in future development. 

“We don’t want our radiologists to automatically do whatever the tool says,” Denney pointed out. “Before our radiologists are allowed to use this tool, they actually have to go through educational training and be very familiar with what all of the best practices are for the different cases. They are also educated on what the truth should be.”

“The tool helps increase their efficiency, but at the same time they’re not just blindly inserting everything that the tool says should be there,” she furthered. 

The technology does not identify every potential finding yet, explained Denney. 

“For example, we’re only looking for incidental lung nodules. We’re not looking for any lung nodules because the treatment plan, depending on what the etiology of the condition is, is going to be different,” she continued.

More than a buzzword search, the machine learning technology is looking for detailed pathology in accordance with Radiology Partner’s criteria for best practice notes. 

Providers will dictate findings as a part of their standard workflow. Natural language processing will then interpret that note and machine learning will identify incidental findings. The algorithm uses a decision tree to produce treatment recommendations to the provider based on best practices. 

“We have all of our radiologists helping to inform what rules and pathology are applicable to these best practices and what are not,” Denney noted. “These best practice recommendations increase the standard of care that we’re providing for our patients by making sure that we’re getting that evidence-based research into our radiology reports. Patients then have the best care and recommendations for treatment that the referring physicians may not always be knowledgeable about.”

Integrating this technology into clinical practice required its implementation to enhance rather than interpret existing provider workflows. 

“We know a lot of our radiologists are not interested in producing structured reports,” Denney noted. “Instead, they’ve been trained to dictate in prose, so they’re not going to change the way that they dictate just because we say we have a tool that could do something for them.”

The easier the tool is for providers to use, the more apt they are to use it. 

“We spend more time working and improving our tool so that it can actually work across the different dictation styles within our practice regardless of what the style is,” she said. 

Implementing this technology into standard practices is really a change management process, Denney argued. 

“Change for anybody is really difficult. We spend a lot of time working with radiologists as we’re rolling this tool out and working with them to understand what their workflow is or to create a tool with as many user experience options as possible to fit into as many workflows as possible,” she emphasized. 

The information gleaned from this roll out will allow for further development of programs. The team can then examine long-term health outcomes for those who underwent treatment recommendations to understand if patient outcomes improved. 

Denney pointed out that this is not the only artificial intelligence Radiology Partners is rolling out, though. Rolling out multiple efforts will allow the team to examine which tools are the most effective at improving patient outcomes and easing provider burden. 

“We have used evidence-based research to help define best practice recommendations for patient’s treatment,” Denney concluded. “We’re not putting all of our eggs in one basket in terms of AI, but we’re very interested in the myriad ways that it can help improve patient care.”