How to Predict the Onset of Anxiety and Depression in Pediatric Residents

Introduction The mental health of medical staff has a significant impact on the quality of medical service work.1 Good mental health of medical staff helps to improve doctor-patient relationships and improve the quality of medical services.2 Some previous studies have shown that medical workers have more severe psychological problems than the general population.3,4 A meta-analysis of 54 published studies suggests that approximately 28.8% of resident physicians worldwide suffer from depression.5 A survey of Chinese doctors showed that 28.12% had depression symptoms, 25.67% had anxiety symptoms, and 19.01% had both depression and anxiety symptoms.6 It is believed that anxiety and depression have become a major cause of mental health hazards for physicians worldwide. Residents face many new challenges as they enter the medical workforce and learn for the first time how to navigate the medical system. They must manage interpersonal relationships with patients, with administrators, and with other colleagues, as well as handle an increasingly demanding hospital workload.7 In recent years, there has been concern about the psychological condition of pediatricians in China, who work with high intensity and risk, but earn less than other departments.8,9 What’s more, pediatricians have a 12.6% turnover rate, most of whom are young.10 Therefore, the mental health of pediatric residents should also be brought to the attention of the public and government officials. According to previous studies, the mental health status of medical staff may be related to the following factors: 1) General sociodemographic information suggests that gender may be an important influencing factor, with some studies suggesting that female medical staff have lower levels of mental health than men and are more likely to experience anxiety and depression.6 The effect of age has not been consistent in previous studies, but low income levels, being unmarried or divorced, and not having children may be high risk factors for psychological problems;3 2) Work stress is an important risk factor for mental health problems in medical staff, as health care workers exposed to high levels of stress are more likely to suffer from burnout11 and emotional-behavioral problems.12 Worry and depression are often the outward manifestations of stress;13 3) Good social support can reduce the mental health problems of individuals and reduce the damage to the physical and mental health of individuals in stressful situations.14 Individuals with low social support are at significantly higher risk for anxiety and depressive symptoms than those with high social support;15 4) Resilience is a set of intrinsic psychological and adaptive traits that protect individuals in adverse, stressful and difficult environments.16,17 Psychological resilience has the effect of promoting mental health and reducing emotional problems such as depression;18,19 5) Other factors, such as public health emergencies,20,21 hospital management system,22 physicians’ specialties,23 and personality traits of physicians2 can also affect the mental health status of medical professionals. Most of the above studies focused on the exploration of factors influencing the mental health status of medical staff.24,25 Early warning prediction requires not only exploring influencing factors, but also utilizing more scientific, accurate, and effective methods to assess and diagnose the mental health status of medical staff in order to identify and intervene in psychological problems in a timely manner. Machine learning, as a powerful technique for data analysis and pattern recognition, is able to extract useful information and knowledge from large amounts of complex data and has been widely used for early warning prediction of psychological health conditions.26–28 Existing studies of machine learning applied to early warning prediction of mental health conditions mostly build evaluation in several models and screen the applied models, ignoring the advantages of different machine learning methods in different fields with different data. Therefore, in this study, we have screened the final early warning prediction models among 18 mainstream machine learning methods according to the no free lunch theory. At the same time, the ML method was applied to medical staff anxiety and/or depression screening by combining socio-demographic information, mood self-rating scale, social support rating scale, and psychological resilience scale, and the model was optimized by feature selection. Finally, the combination of using discrimination and calibration to evaluate and screen the optimal models has also provided a new perspective on early warning prediction of anxiety and/or depression for medical professionals. Methods Study Design and Population This study conducted questionnaires from January 2022 to March 2023 at the Children’s Hospital of Zhejiang University School of Medicine and the Children’s Hospital of Chongqing Medical University, both of which are national regional pediatric medical centers in China and have the highest number of pediatric residents in China. A total of 138 online questionnaires (all valid) were collected from Hangzhou, Zhejiang Province, and 173 online questionnaires (171 of which were valid) from Chongqing under the premise of voluntary completion. The questionnaire included socio-demographic information (gender, age, years of work, education, annual income, physician category, practice direction, etc)., emotional self-rating scale, social support rating scale, and psychological resilience scale. Findings focused on anxiety and/or depression, with sociodemographic information and components such as scale entries, subscale scores, and total scale scores included as study variables. The questionnaires used in this study are based on previously published literature, and additionally, related research demonstrates the relationship between social support and negative emotions among pediatric residents.29 Emotional Self-Assessment Scale The scale consists of three subscales: anxiety, depression, and stress,30 each containing 7 items, for a total of 21 items, on a four-point scale of 0 to 3.31 Stress subscale scores from this scale were included as study variables, and findings were focused on scores on the anxiety and depression subscales; scores were rated as positive for anxiety-depression if anxiety and/or depression scores were abnormal, and as negative for anxiety-depression if both anxiety and depression scores were normal. The total Cronbach’s alpha value of the scale was 0.929, and the Cronbach’s alpha values of the three subscales were 0.880, 0.879, 0.892, respectively. Social Support Rating Scale (SSRS) The social support rating scale developed by Chinese scholar Xiao Shuihui was used. The scale has 10 items, including three dimensions of objective support (three items), subjective support (four items), and utilization of social support (three items), and overall, the higher the score, the higher the individual’s level of social support.32 In this study, the SSRS overall Cronbach’s alpha value was 0.940. Psychological Resilience Scale The psychological resilience scale (Connor-Davidson Resilience Scale, CD-RISC) jointly developed by Connor and Davidson was used in the current study.16 The scale consists of 25 items, which are assessed using the LIKERT 5-point scale, with higher scores indicating higher elasticity.19 Some scholars have divided the scale into three dimensions of resilience, strength, and optimism.33 In this study, the CD-RISC overall Cronbach’s alpha value was 0.962. Prediction Models Eighteen mainstream machine learning algorithms were incorporated into model training, encompassing common machine learning classifiers: Logistic Regression, Gradient Boosting Classifier, Linear Discriminant Analysis, Quadratic Discriminant Analysis, Ada Boost Classifier, Naive Bayes, Decision Tree Classifier, Light Gradient Boosting Machine, Extra Trees Classifier, Random Forest Classifier, Dummy Classifier, K Neighbors Classifier, SVM-Linear Kernel, Ridge Classifier and so on. The machine learning models with the best overall performance on the training set were included in the subsequent model evaluation.2.3 Model Derivation, Internal and External Validation. Model Performance Evaluation and Interpretability Analysis The Chongqing data was used for model building and internal validation, and the data was divided into a 7:3 training set and a test set, while the Hangzhou data was used for external validation to explore the generalization capability of the model. In the modeling phase, data preprocessing and feature engineering are performed on the training set to avoid data leakage. Data pre-processing includes: 1. use maximum-minimum normalization, 2. exclude missing values and zero variance variables. The choice of variables is decisive for the performance of the final mo therefore, this study used the cross-validated LASSO method to deal with the list of variables. The cross-validated LASSO method is a feature selection method using cross-validation with the LASSO classifier. The hyperparameters of the model were selected using grid search with 10-fold cross-validation. Model differentiation evaluation was assessed by accuracy, AUCroc, recall, precision, F1-score, kappa. The calibration of the models was tested using a goodness-of-fit test to assess the consistency of the predicted results with the true situation. Finally, external validation data were used to explore the generalization ability of the models on unknown data sets. In addition, we use the SHapley Additive exPlanations (SHAP) method to assess the interpretability of the best models and focus on the main effects and interactions of key features. The SHAP method is derived from game theory based on SHAP values, which shows the importance of variables and identifies the direction of effects. Statistical Analysis Concentration and dispersion trends for continuous variables are expressed as mean ± standard deviation (normally distributed with chi-squared variance) and median of interquartile range (IQR, skewed distribution), and categorical variables are expressed as percentages. The χ2 test or Fisher’s exact test was used to compare rates, and the t test was used to compare means with the Wilcox rank sum test. A two-sided P<0.05 was considered statistically significant. This study was implemented using python 3.8 with R 4.2.0, and the random seed was set to 123. Model construction, tuning, interpretability evaluation and partial discrimination evaluation were implemented using PyCaret 2.3.10. Confidence intervals for AUC were calculated using multiROC 1.1.0. Results Participants Characteristics A total of 171 people were included in the Chongqing survey site, of which the median age was 26.0 (25.0, 27.0), the respondents were predominantly female 120 (70.18%), and the educational level was predominantly master 88 (51.46%). Survey respondents without anxiety and depression had higher scores on subjective support, support utilization, total social support score, stress subscale score, total psychological resilience score, and each of its subscales, with statistically significant differences, as shown in Table 1. A total of 138 pediatric residents in Hangzhou were included, including a median age of 26.0 (24.0, 27.8), and respondents without anxiety and depression had higher support utilization, total social support score, stress subscale score, total psychological resilience score, and scores on each of their subscales, with statistically significant differences, as detailed in the Supplementary Table 1 and the Supplementary Table 2. Table 1 Descriptive Analysis of Chongqing Data Selection of Predictors Using LASSO and Selection of Model Feature selection as well as model selection were performed in the training set, and 6 features were finally selected based on the LASSO-CV method among a total of 82 variables of sociodemographic information, social support rating scale, emotional self-rating scale, and psychological resilience scale to construct the subsequent model, as shown in the Supplementary Table 3, the Supplementary Table 4 and the Supplementary Figure 1. The discriminant index of the models was calculated using the screened features with a 10-fold cross-validation and the top ten models with AUC were ranked. The top three models with AUC were screened to plot the calibration graphs, see Table 2 and the Supplementary Figure 2. The Linear Discriminant Analysis (LDA) model was selected for the final modeling by combining the discrimination and calibration degrees. Model Performance Between Models in Internal and External Validation The LDA is tuned using grid search on the training set, and the performance and generalization ability of the model are evaluated on the Chongqing test set and Hangzhou external validation dataset. The tuned LDA model has both excellent discriminatory performance (AUC=0.923) on the test set, as shown in Table 3. On the external dataset where the generalization ability of the model was evaluated, it still showed excellent discriminative performance (AUC=0.935). Variable Importance for Prediction Models The final screened LDA model had extremely high discriminatory performance in distinguishing the presence of anxious depressive mood in pediatric residents. To further explain the black box problem of the LDA model, we used the SHAP method to evaluate the interpretability of the LDA model. The SHAP results showed that the top three significant predictors were stress subscale scores, strength (a subscale of the psychological resilience scale), and total psychological resilience scale scores, as shown in Figure 1. The predictive impact of the variables on anxiety-depression was mainly unidirectional, such that an increase in stress scores, low strength, low psychological resilience scale scores, and an increase in age increased the likelihood of anxiety-depression among pediatric residents, as shown in Figure 2. Figure 1 Variable importance evaluation of LDA. Discussion This study investigated the psychological status of pediatric residents in two urban children’s hospitals and utilized an LDA model evaluated by feature selection and performance (discrimination and calibration), with results suggesting good discrimination performance. The feature selection results showed that sociodemographic correlates (eg, gender, annual income, etc.) were considered as exclusion variables in the LASSO feature selection, which also differed from some previous correlation analyses. The results of the performance evaluation and interpretability analysis showed that the stress, psychological resilience (especially tenacity) of pediatric residents were the most important predictors of the occurrence of anxiety and depression in this group. The results of validation on external data confirmed the robustness of this model. Thus, it further confirms that stress, psychological resilience is a key predictor in this study. The main sources of work stress for physicians include long working hours,34 lack of organizational resources (eg, low staffing, shortage of equipment, etc)., and heavy workload.35 Exposure to uncontrollable stress can have deleterious effects on the prefrontal cortex (PFC), the area of the brain that controls higher cognition and provides top-down control of thoughts, actions, and emotions,36–38 which may be the neural mechanism by which stress leads to anxious depressed mood. It is widely accepted that occupational stress can lead to burnout.39,40 Burnout may co-occur with anxiety41 and may be one of the key factors in the development of anxiety and depression in pediatric residents.29 Pediatric residents in China work on the front lines with high workloads but low incomes, and this effort-reward imbalance (ERI) can affect health, with a meta-analysis of more than 80,000 people showing that staff exposed to ERI are at significantly increased risk of depression.42 Burnout was not introduced as a variable in this study, mainly because there is a contemporary tendency to equate burnout with anxiety-depression in medical personnel, which is not the case; not all burnout leads to anxiety-depression, and not all anxiety-depression comes from burnout, although there are similarities in their external manifestations, but timely and accurate identification of anxiety-depression states in pediatric residents is very relevant to obtain appropriate treatment. Psychological resilience is the ability or trait that allows individuals to adapt effectively in the face of life adversities such as stress, setbacks, and trauma,43 and it is one of the important psychological capital and an important psychological resource.44 The study revealed that low psychological resilience (especially low tenacity) in pediatric residents was a strong early warning predictor of anxiety and depression, which is consistent with the three-system mechanism model of psychological resilience proposed by Davydov45 that psychological resilience influences mental health through harm reduction, protection from negative influences, and improved problem-solving skills. It is evident that the primary role of psychological resilience is to reduce the negative effects of depression and anxiety brought about by unfavorable situations. Therefore, strengthening psychological resilience through psychological interventions has become a hot topic of research in recent years. Some scholars believe that interventions to enhance psychological resilience can be carried out from three aspects: the individual himself/herself, the family, and the social environment.46 Not only is positive thinking one of the personal factors influencing psychological resilience, but it is also an important predictor of psychological resilience,47 and higher positive thinking is associated with lower emotional reactions, faster return to baseline after exposure to negative stimuli, and more comprehensive positive cognition,48 which can facilitate individuals to further construct lasting psychological and social resources.49 Thus, positive thinking training is currently a more applied approach. Exploration of positive thinking training in physician populations50–52 has also begun, with only mixed evaluations of effectiveness and extremely limited research on positive thinking training among pediatric residents. This study also found that pediatric residents have “age anxiety”, which may be related to the long training cycle of Chinese physicians, with at least 5 years of medical school, and possibly 8 years (for a master’s degree in clinical medicine) and 11 years (for a PhD in clinical medicine), making the age of medical school graduation older than other specialties, and most medical school graduates have to participate in government-led residency training to become a resident, and the income of junior residents is generally not high. The pressure of marriage, childbirth, and buying a house increases with age, and a rigorous qualitative interview study is needed to obtain a clear association with “age”. Objective support in social support, ie, the practical support received, including the comfort of loved ones and direct material assistance in case of difficulties, etc., and strengthening this aspect of support for pediatric residents may have positive implications for reducing their anxiety and depression. The present study has some shortcomings. First, the data in this study comes from only two cities, which affects population diversity, may lead to potential bias, affecting the model’s generalization ability and potentially impacting the accuracy of predictions. Therefore, further validation in more diverse or external populations, or prospective testing, is necessary to verify the effectiveness of the predictive model. Second, this is a cross-sectional study. Therefore, the chronological order of causality is impossible to determine the sequence of occurrence between the change in variables (increased stress and psychological resilience) and the change in the outcome of this study (increased likelihood of anxiety and depression). The above results remain somewhat indicative of reality. While medical workers are still medical students, their emotional problems may be first visible (29% of medical students showed symptoms of depression and 21% of medical students showed symptoms of anxiety).53 In future studies, the establishment of a medical student/physician mental health follow-up cohort to monitor physician mental health status longitudinally would be valuable. Furthermore, to clarify the significance of stress relief and enhanced psychological resilience in mitigating anxiety and depression, a randomized controlled psychological intervention trial(RCT) is the gold standard. In the next step, our research team will also conduct corresponding psychological intervention trials to explore solutions for the negative emotions of pediatric residents. Conclusions In conclusion, this study provides a new perspective on early warning prediction and intervention for negative emotions among pediatric residents and medical staff, which may be a new direction for intervention to address mental health issues in this population. Our findings suggest the use of six key predictor variables that can play a critical role in pediatric residents’ and medical staff’s negative emotions. Moving forward, integrating this six-key predictor variable screening tool into hospitals’ annual resident surveys will translate the validated predictive model into actionable real-world practice. Machine learning and SHAP methods provide powerful tools for identifying complex influencing factors of negative emotions among resident physicians, but their clinical translation requires careful threshold setting, ethical considerations, and system integration. By establishing multi-level intervention triggering mechanisms, addressing algorithmic bias issues, ensuring clinical transparency, and granting doctors ultimate decision-making power, these tools are expected to become important supplements to mental health services, ultimately achieving a shift from passive treatment to predictive and preventive mental health management.