Researchers Collaborate to Advance Machine Learning for RA Patients

November 10, 2021 - At the American College of Rheumatology (ACR) annual meeting on November 6, Hospital for Special Surgery researchers presented the results from the RA2-Dream Challenge, a crowdsourced effort to develop machine learning tools to qualify joint damage in individuals with rheumatoid arthritis (RA).

Damage in the joints of those with RA is currently measured by visual inspection and detailed scoring on radiographic images of small joints in the hands, wrists, and feet, including both joint space narrowing and bone erosions. However, according to researchers, the scoring system requires specially trained experts and can be time-consuming and expensive.

Using artificial intelligence approaches to measure joint damage could be critical in improving both clinical research and patient care.

“If a machine-learning approach could provide a quick, accurate quantitative score estimating the degree of joint damage in hands and feet, it would greatly help clinical research,” the study’s senior author S. Louis Bridges, Jr., MD, PhD, said in a press release.

“For example, researchers could analyze data from electronic health records and from genetic and other research assays to find biomarkers associated with progressive damage. Having to score all the images by visual inspection ourselves would be tedious, and outsourcing it is cost-prohibitive.”

According to Bridges, the machine learning method could also assist rheumatologists by quickly evaluating damage progression over time, providing insight into treatment options. “This is really important in geographic areas where expert musculoskeletal radiologists are not available,” Bridges added.

For the challenge, Bridges and his team partnered with Sage Bionetworks, a nonprofit organization that helps researchers develop DREAM (Dialogue on Reverse Engineering Assessment and Methods) Challenges. These competitions focus on creating artificial intelligence tools in the life sciences. The researchers sent out a call for submissions, with grant money providing prizes to the winning teams.

“For the first part of the challenge, one set of images was provided to the teams, along with known scores that had been visually generated. These were used to train the algorithms. Additional sets of images were then provided so the competitors could test and refine the tools they had developed,” the press release stated.

“In the final round, a third set of images was given without scores, and competitors estimated the amount of joint space narrowing and erosions. Submissions were judged according to which most closely replicated the gold-standard visually generated scores.”

A total of 26 teams submitted algorithms and 16 final submissions. Competitors were given 674 sets of images from 562 different RA patients. The data was collected from patients who had participated in a previous National Institutes of Health research study. Four teams were named top performers.

For the DREAM Challenge organizers, any developed scoring system needed to be widely accessible and free. “Part of the appeal of this collaboration was that everything is in the public domain,” Bridges said.

Bridges explained that additional research and development of AI methods are needed before the tools can be used broadly. However, current research suggests that this type of approach is feasible. “We still need to refine the algorithms, but we’re much closer to our goal than we were before the Challenge,” Bridges said.