ML Model Accurately Predicts 6-Month Mortality Risk for Cancer Patients

August 18, 2023 - In a study recently published in JAMA Network Open, researchers have externally validated a machine learning (ML) model designed to predict six-month mortality risk for patients with advanced cancer initiating a new line of therapy (LOT).

The model was originally developed and internally validated to classify patients into mortality risk categories in order to better facilitate effective serious illness conversations between providers and patients at various treatment decision points (TDPs).

However, the researchers indicated that external validation of any healthcare ML model is key for ensuring trustworthiness prior to clinical deployment. Despite this, results of external validation studies are rarely reported, spurring the research team to externally validate their model on recent patient data.

The model uses 45 features pulled from EHR data that are implementable via the Fast Healthcare Interoperability Resources (FHIR) standard. The researchers further noted that the algorithm is expected to be integrated into a tool to help communicate and explain ML-driven prognosis.

The model was originally trained on patient TDP data from between June 1, 2014, and June 1, 2020. Using this information, the research team identified TDPs for new LOTs, then confirmed mortality at six months following each TDP.

External validation was conducted using newly identified TDPs between June 2, 2020, and April 12, 2022.

To evaluate the model, the researchers compared population characteristics between the development and validation data, in addition to assessing overall performance using area under the curve (AUC), positive predictive value, negative predictive value, sensitivity, and specificity at a predetermined risk threshold of 0.3.

This risk threshold was selected to be consistent with the previous study, in which patients were classified into low or higher chance of survival, and approximately 1 in 3 patients classified as low chance of surviving were alive after six months.

For patients identified as having a low chance of survival, multiple quality metrics, including referrals for palliative care or hospice, hospitalization rates, and mean length of stay were calculated.

The 1,822 patients in the validation cohort experienced 2,613 TDPs. The development and validation data were similar, with no significant differences in six-month mortality rates following TDPs. However, patients in the validation dataset tended to be younger, with a higher proportion of nervous system and brain cancer, but a lower proportion of lung cancer compared to the development dataset.

The model achieved high performance, with an AUC of 0.80 for the validation cohort.

Low chance of survival was flagged in 8.7 percent of TDPs, and quality metrics across this cohort varied. For the 146 TDPs among 130 patients forecasted to have a low chance of survival who died within six months, 16.4 percent had hospice referral, 49.3 percent had palliative care referral, and 64.4 percent had a hospitalization between TDP and death.

The research team concluded that these findings support the need for a tool that can help facilitate serious illness conversations for providers and patients considering new anticancer LOTs. However, the research team cautioned that the model’s generalizability is limited by the study’s single-center datasets and lack of racial and ethnic diversity within the study cohorts.

They further noted that while the study provides an important quality check prior to the model’s potential integration into oncology care, but that additional validation across multiple health systems is needed.

Others are also looking to ML to help predict cancer mortality and improve care.

In 2022, researchers demonstrated that an ensemble of ML models can accurately forecast six-month ovarian cancer mortality using patient-reported outcome (PRO) data.

The research team indicated that five-year survival rates for ovarian cancer vary significantly based on cancer stage and type, with late-stage ovarian cancers having some of the lowest survival rates.

Further, with recurrent or late-stage disease, ovarian cancer treatment can come at the expense of patient quality of life, as many treatments are associated with significant side effects.

To improve clinical decision-making and end-of-life care, the researchers developed their model to identify when an ovarian cancer patient is reaching the end of their life using PROs like symptom severity and interference, depression, anxiety, health-related quality of life, and cancer-related quality of life.

The predictive model accurately identified a majority of patients who died within 180 days of PRO assessment, indicating that such a tool could address shortcomings in ovarian cancer care delivery.