Machine Learning Flags Preoperative Anxiety in Hip Fracture Patients

Three machine learning models that used routine medical and psychological data were similarly accurate at identifying anxiety before surgery in patients with broken femoral necks, according to a study that looked back at patient records from one hospital in China. The researchers studied 271 hospital admissions involving 270 patients who had surgery. The three models were extreme gradient boosting (XGBoost), random forest, and Light Gradient Boosting Machine (LightGBM). Their areas under the receiver operating characteristic curve (AUC) were 0.921, 0.932, and 0.928, respectively. This measure shows how well a model can distinguish between patients with and without anxiety. The confidence intervals for the results overlapped widely, suggesting that the models performed similarly. The study used records from the Second Affiliated Hospital of Wenzhou Medical University from January 1 to December 31, 2024. The researchers wanted to find out whether combining physical health, psychological, and social information could identify anxiety measured by a validated screening tool. They also wanted to see which factors remained important across repeated tests. The authors stressed that the models classified anxiety during the pre-surgery assessment period. They did not predict a separate outcome that happened later. None of the models has been tested in a new group of patients or in a prospective study that follows patients over time.

The researchers began with 418 hospital admission records for adults who had imaging-confirmed acute femoral neck fractures and were scheduled for surgery. They removed 24 records involving fractures caused by disease or multiple serious injuries from high-energy accidents, 15 involving severe mental health conditions or long-term use of psychiatric medicines, 65 involving severe problems with thinking or memory, and 43 with missing key medical or questionnaire information. This left 271 admissions for analysis. Researchers measured anxiety using the anxiety section of the Hospital Anxiety and Depression Scale (HADS-A). Trained researchers asked patients questions in face-to-face bedside interviews 24 to 48 hours after admission and at least 12 hours before surgery. Using a cutoff chosen in advance, a HADS-A score above 8 meant that a patient had significant anxiety before surgery. Based on this rule, 79 admissions (29.2 percent) were classified as having significant anxiety, while 192 (70.8 percent) were not. The data used for analysis kept only these two categories, not the actual HADS-A scores or individual answers. As a result, the researchers could not classify patients using different cutoffs or check how changing the cutoff affected the results based on the full scores.

Comparisons at the start of the study showed several differences between the groups. Patients in the anxiety group were older on average, with a mean age of 70.39 years, compared with 65.48 years among those without anxiety. Their average albumin level, a measure related to nutrition and health, was also lower: 39.25 versus 40.66 grams per litre. Pain scores before surgery, measured on a visual analogue scale, were not significantly different. The average was 3.81 in the anxiety group and 3.59 in the other group. The psychological and social measures showed clearer differences. Average scores on the Connor-Davidson Resilience Scale (CD-RISC), which measures the ability to cope with difficulties, were 48.66 among patients with anxiety and 64.36 among those without it. Scores on the Multidimensional Scale of Perceived Social Support (MSPSS) were 48.05 versus 57.80. Scores on the Surgical Fear Questionnaire (SFQ) were 11.47 versus 8.08. All three differences had p values below 0.001, meaning they were highly unlikely to be explained by random variation under the statistical test's assumptions. The anxiety group also had a higher frailty index, at 0.26 compared with 0.20. Only 32.9 percent of admissions in the anxiety group involved familiar people being present when the patient arrived, compared with 53.6 percent of admissions in the group without anxiety.

The researchers grouped the information they studied into three areas: physical health, psychological factors, and social circumstances. Physical health information included frailty and other illnesses. Psychological information included resilience and fear of surgery. Social information included perceived support from other people and whether familiar people were present when the patient was admitted. Medical records also supplied information about age, gender, body mass index, haemoglobin, albumin, the Charlson comorbidity index, the American Society of Anesthesiologists (ASA) grade, the time between injury and admission, previous surgery and anaesthesia, and pain before surgery. Administrative details, such as the order of cases, the time patients entered the operating room, and patient identifiers, were left out. The researchers also excluded the anxiety outcome itself from the predictors. This left 32 possible numeric predictors. None of the predictor or outcome values was missing, so a median-based method included in the analysis process did not need to fill in any missing values.

The researchers designed their testing process to reduce the risk of the models appearing more accurate than they really were. They used least absolute shrinkage and selection operator (LASSO) logistic regression to choose which features to keep. The method's regularisation setting, which controls how many features are retained and how strongly their effects are reduced, was tuned inside each inner training group. The entire feature-selection process was repeated separately in each of five outer folds, rather than forcing every test to use one fixed set of features. Each inner pipeline also recalculated standardised values and applied the synthetic minority oversampling technique (SMOTE), which helps address differences in the number of patients with and without anxiety, before fitting a model. The outer validation used fivefold stratified group cross-validation, with patient identifiers used to keep records from the same patient together. This ensured that records from one patient could not appear in both the training and test data. Within each outer training group, about 75 percent of the records were used to fit the model and 25 percent were used to adjust its probability estimates through Platt sigmoid calibration. The researchers tested 18 randomly selected combinations of model settings in each fold. The outer test group was never used to prepare the data, select features, balance the groups, tune settings, or calibrate probabilities.

The three algorithms performed so similarly that the study found no clear statistical difference between them. Random forest had the highest estimated AUC at 0.932, with a 95 percent confidence interval of 0.900 to 0.959. LightGBM followed at 0.928 (0.894 to 0.957), and XGBoost scored 0.921 (0.885 to 0.953). Paired bootstrap tests comparing the models' AUC scores all produced confidence intervals that included zero, meaning the tests did not show a clear difference. The difference between random forest and XGBoost was 0.011 (95 percent confidence interval -0.011 to 0.033). The difference between LightGBM and XGBoost was 0.007 (-0.013 to 0.029), while the difference between LightGBM and random forest was -0.004 (-0.027 to 0.019). Brier scores, which measure the accuracy of predicted probabilities, were also close, ranging from 0.096 to 0.099. For random forest, the calibration intercept was 0.041 (95 percent confidence interval -0.325 to 0.386), and the calibration slope was 1.017 (0.690 to 1.571). The calibration results for XGBoost and LightGBM were similarly consistent with acceptable calibration, although their confidence intervals were wide. The authors concluded that the overlapping confidence intervals and paired comparisons did not show that any one algorithm was better than the others in this dataset.

The feature-selection results were especially consistent for the psychological and social measures. The LASSO method selected CD-RISC, MSPSS, and SFQ in all five outer folds for all three models. It selected the frailty index in three of the five folds for each model. The pain score before surgery was not significantly different between the groups in the initial comparison, but it was selected in three of five folds for XGBoost and LightGBM and two of five folds for random forest. The researchers said this might mean pain added useful information when considered alongside other variables. However, they cautioned that this does not prove pain has an independent relationship with anxiety. The researchers also used TreeSHAP to examine how features contributed to XGBoost's results on data held out during cross-validation. The four features with the largest average absolute contributions were CD-RISC, MSPSS, SFQ, and the frailty index. Higher resilience and social support scores generally lowered the model's estimated probability of anxiety. Higher surgical fear scores, and less consistently higher frailty scores, raised it. The authors warned that SHAP values explain how a model makes its classifications. They do not show that a factor causes anxiety or prove that the model is clinically useful.

An additional exploratory analysis divided admissions into three groups based on the anxiety probabilities estimated by the random forest model for records held out during cross-validation. Among the 91 admissions in the low-probability group, none had observed anxiety: 0.0 percent, with a 95 percent confidence interval of 0.0 to 4.1 percent. In the middle group, 13 of 90 admissions had anxiety, or 14.4 percent (95 percent confidence interval 8.6 to 23.2 percent). In the high-probability group, 66 of 90 admissions had anxiety, or 73.3 percent (95 percent confidence interval 63.4 to 81.4 percent). The estimated probability ranges were 0.0000 to 0.0435 for the low group, 0.0438 to 0.3687 for the middle group, and 0.3688 to 1.0000 for the high group. The researchers stressed that these three groups depend on this particular sample and describe how the model ranked admissions within the study. They are not clinically validated thresholds for making treatment decisions. The decision curve analysis and results for these three groups used 2,000 bootstrap samples grouped by patient to estimate uncertainty.

The authors clearly described several important limits to the study. It looked back at records from just one hospital and included only 79 cases of anxiety. This small number of cases, the unequal numbers of patients with and without anxiety, and the need to calibrate each fold separately could still lead to overfitting, even with the nested testing process. The original records did not preserve the exact time when every possible predictor was collected. HADS-A scores and several psychological and social measures were also collected during the same pre-surgery assessment period. Therefore, the models cannot show that all their inputs would be available at some later point when a decision needs to be made. The researchers did not assess how well the models would work if used with data from a different setting, whether their probabilities were well calibrated for different patient groups, or whether the algorithms might produce biased results. The models have not been tested in other hospitals or in prospective studies. As a result, the findings do not prove that using the models would improve clinical care, nursing workflows, or patient outcomes. The authors recommend that future research collect data prospectively and specify in advance when each predictor becomes available. Future studies should keep the total HADS-A scores and individual answers, test the models in outside populations, check how well they distinguish anxiety and estimate probabilities, examine performance across patient groups and possible algorithmic bias, and test whether nursing assessments guided by the models improve outcomes that matter to patients. Until this work is completed, the authors say that healthcare staff should continue to assess anxiety using validated screening tools and clinical judgement.

Subject of Research: Using explainable machine learning to classify anxiety before surgery in patients with femoral neck fractures

Article Title: Explainable Machine Learning Models for Classifying Preoperative Anxiety in Patients With Femoral Neck Fractures: A Retrospective Cohort Study

Article References: Zhang, X., Zheng, S., Zhang, Z., Yang, P., Wei, Q., Jin, M., & He, F. (2026). Explainable Machine Learning Models for Classifying Preoperative Anxiety in Patients With Femoral Neck Fractures: A Retrospective Cohort Study. Nursing Open, 13(10), Article e70919. https://doi.org/10.1002/nop2.70919

Image Credits: AI Generated

DOI: 10.1002/nop2.70919

Keywords: machine learning, anxiety before surgery, femoral neck fracture, XGBoost, random forest, LightGBM, SHAP, nested cross-validation, HADS-A, biopsychosocial model, frailty, nursing