Ling Tong, PhD

Selected publications

1. 

   Socioeconomic Disparities for Healthcare Utilization of Senior Adult Falls in Southeast Wisconsin, 2020-2022

   Tong, Ling, Khani, Masoud,  Taylor, Bradley and 4 more authors

   medRxiv Apr 2023

   Bib HTML

   @article{Tong2023socioeconomic,
     author = {Tong, Ling and Khani, Masoud and Taylor, Bradley and Osinski, Kristen and Adams, Jazzmyne and Friedland, David and Luo, Jake},
     title = {Socioeconomic Disparities for Healthcare Utilization of Senior Adult Falls in Southeast Wisconsin, 2020-2022},
     year = {2023},
     month = apr,
     doi = {10.1101/2023.04.24.23289062},
     publisher = {Cold Spring Harbor Laboratory Press},
     journal = {medRxiv},
   }

2. 

   Interpretable Machine Learning Text Classification for Clinical Computed Tomography reports – A Case Study of Temporal Bone Fracture

   Tong, Ling, Luo, Jake,  Adams, Jazzmyne and 3 more authors

   Computer Methods and Programs in Biomedicine Update. Apr 2023

   Abs Bib HTML

   Background: Machine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model’s complexity, interpreting mechanisms of classification results remains difficult. Methods: We investigated interpretable representations of machine learning classification models. We created machine learning models to classify temporal bone fractures based on 164 temporal bone Computed Tomography (CT) text reports. We adopted the XGBoost, Support Vector Machine, Logistic Regression, and Random Forest algorithms. To interpret models, we used two major methodologies: (1) We calculated the average word frequency score (WFS) for keywords. The word frequency score shows the frequency gap between positive and negative classified cases. (2) We used Local Interpretable Model-Agnostic Explanations (LIME) to show the word-level contribution to bone fracture classification. Results: In temporal bone fracture classification, the random forest model achieved an average F1-score of 0.93. WFS reveals a difference in keyword usage between fracture and non-fracture cases. Additionally, LIME visualized the keywords’ contributions to classification results. The evaluation of LIME-based interpretation achieved the highest interpreting accuracy of 0.97. Conclusion: The interpretable text explainer can improve physicians’ understanding of machine learning predictions. By providing simple visualization, our model can increase the trust of computerized models. Our model supports more transparent computerized decision-making in clinical settings.

   @article{tong2022interpretable,
     title = {Interpretable Machine Learning Text Classification for Clinical Computed Tomography reports – A Case Study of Temporal Bone Fracture},
     author = {Tong, Ling and Luo, Jake and Adams, Jazzmyne and Liu, Xiaoyu and Osinski, Kathleen and Fridland, David},
     year = {2023},
     month = apr,
     journal = {Computer Methods and Programs in Biomedicine Update.},
   }

3. 

   Association between Body-Mass Index, Patient Characteristics, and Obesity-related Comorbidities among COVID-19 Patients: A Prospective Cohort Study

   Tong, Ling, Khani, Masoud,  Lu, Qiang and 3 more authors

   Obesity Research and Clinical Practice Dec 2022

   Abs Bib HTML

   Objective: Obesity is a major risk factor for adverse outcomes after COVID-19 infection. However, it is unknown if the worse outcomes are due to the confounding effect of demographic and obesity-related comorbidities. The study objective is to analyze associations between body mass index, patient characteristics, obesity-related comorbidity, and clinical outcomes in COVID-19 patients. Methods: In this prospective cohort study, we chose patient records between March 1st, 2020, and December 1st, 2022, in a large tertiary care center in southeast Wisconsin in the United States. Patients over the age of 18 who tested positive were included in the study. Clinical outcomes included hospitalization, intensive care unit (ICU) admission, mechanical ventilation, and mortality rates. We examined the characteristics of patients who had positive clinical outcomes. We created unadjusted logistic regression models, sequentially adjusting for demographic and comorbidity variables, to assess the independent associations between BMI, patient characteristics, obesity-related comorbidities, and clinical outcomes. Results: From a record of 1.67 million inpatients and outpatients at Froedtert Health Center, 55,299 (BMI: 30.5±7.4 kg/m2, 62.5% female) tested COVID-19 positive during the study period. 17,580 (31.8%) patients were admitted to hospitals, and 2,856 (16.2%) of hospitalized patients required ICU admission. 1038 (36.3%) required mechanical ventilation, and 462 (44.5%) died after a positive test for COVID-19. We found female patients show a higher hospitalization rate, while male patients have a higher rate of ICU admission, mechanical ventilation, and mortality. Obesity-related comorbidities are associated with worse outcomes compared to simple obesity without comorbidities. In logistic regression models, we found four similar V-shaped associations between BMI and four clinical outcomes. Patients with a BMI of 25 kg/m2 are at the lowest risk for clinical outcomes. Patients with a BMI lower than 18 kg/m2 or higher than 30 kg/m2 are associated with a higher risk of hospitalization, ICU, mechanical ventilation, and death. After adjusting the model for demographic factors and hypertension and diabetes as two common comorbidities, we found that demographic factors do not significantly increase the risk. Obesity alone does not significantly increase the risk of severe clinical outcomes. Obesity-related comorbidities, on the other hand, resulted in a significantly higher risk of outcomes. Conclusion: Obesity alone does not increase the risk of worse clinical outcomes after COVID-19 infection. It may suggest that the worse clinical outcomes of patients with obesity are mediated via hypertension and type 2 diabetes. Patients with obesity and comorbidities have a higher risk of poor outcomes. Obesity-related comorbidities, including hypertension and diabetes, are independently associated with poorer clinical outcomes among COVID-19 patients. At a BMI of more than 30 kg/m2 or less than 18 kg/m2, we found an increase in the risk of severe COVID-19 outcomes leading to hospitalization, ICU, mechanical ventilation, and death. The increased risk of severe outcomes is not attributed to patient characteristics but can be attributed to hypertension and diabetes.

   @article{tong2022association,
     title = {Association between Body-Mass Index, Patient Characteristics, and Obesity-related Comorbidities among COVID-19 Patients: A Prospective Cohort Study},
     author = {Tong, Ling and Khani, Masoud and Lu, Qiang and Taylor, Bradley and Osinski, Kristen and Luo, Jake},
     year = {2022},
     month = dec,
     journal = {Obesity Research and Clinical Practice},
   }

4. 

   Visualizing the Interpretation of a Criteria-Driven System that Automatically Evaluates the Quality of Health News: An Exploratory Study of Two Approaches

   Liu, Xiaoyu, Alsghaier, Hiba,  Tong, Ling and 2 more authors

   JMIR AI Nov 2022

   Abs Bib HTML

   Background: Machine learning techniques have been shown efficient at identifying health misinformation. However, interpreting a classification model remains challenging due to the model’s intricacy. The absence of a justification for the classification result and disclosure of the model’s domain knowledge may erode end-users’ trust in such models. This diminished trust may also undermine the effectiveness of artificial intelligence-based initiatives to counteract health misinformation. Objective: The study objective is to address the public’s need for help evaluating the quality of health news and the typical opaqueness of an AI approach. This work aims to create an interpretable, criteria-driven system. We also aim to find the best approach by comparing the accuracy of two methods for selecting and visualizing the system’s interpretation. Both methods highlight the sentences that contributed to the analysis performed by the system for a given criterion. Methods: We employ an interpretable, criteria-based approach for automatically assessing the quality of health news on the Internet. One of ten well-established criteria was chosen for our experimentation. To automate the evaluation of the criterion, we tested Logistic Regression, Naive Bayes, Support Vector Machine, and Random Forest algorithms. We then experimented with two approaches for developing interpretable representations of the results. For the first approach, (1) we calculate word feature weights, which explains how classification models distill keywords that are relevant to the positive prediction;(2) then using the Local Interpretable Model Explanations (LIME) framework, we automatically select keywords for visualization that show how classification models identify positive news articles, based on related keywords;(3)and finally, the system highlights target sentences containing keywords to justify the criterion evaluation result. For the second approach, (1) we extract sentences that provide evidence to support the evaluation result from 100 health news articles, (2) based on these results, we train a typology classification model at a sentence level; (3) then, the system highlights positive sentence instances for the result justification. We assess the accuracy of both methods using a small held-out test set. Results: The Random Forest model achieved the highest average AUC score of 0.74 in evaluating the criterion. Additionally, LIME provided a visualization of how keywords impact the system’s evaluation results. Both approaches could successfully visualize the interpretation of the system. When tested with four cases, the hybrid approach performs slightly better in highlighting correct sentences with an accuracy of 55.00%, compared to 50.00% by the typology approach. Conclusions: We provide an interpretable, criteria-based method to evaluate health news. The method incorporates rule-based and statistical machine learning approaches. Results suggest one might automate criterion-based health news quality evaluation successfully using either approach, but larger differences may arise when multiple quality-related criteria are considered. This work can increase public trust in computerized health information evaluation

   @article{liu2022visualizing,
     title = {Visualizing the Interpretation of a Criteria-Driven System that Automatically Evaluates the Quality of Health News: An Exploratory Study of Two Approaches},
     author = {Liu, Xiaoyu and Alsghaier, Hiba and Tong, Ling and Ataullah, Amna and Mcroy, Susan},
     year = {2022},
     month = nov,
     journal = {JMIR AI},
   }

5. 

   Telemedicine and Health Disparities: Association between Patient Characteristics and Telemedicine, In-person, Telephone and message-based Care During the COVID-19 Pandemic

   Tong, Ling, George, Ben,  Crotty, Bradley and 4 more authors

   IPEM Translation. Sep 2022

   Abs Bib HTML

   Telemedicine has been an essential form of care since the onset of the COVID-19 pandemic. However, telemedicine may exacerbate disparities for populations with limited digital literacy or access, such as older adults, racial minorities, patients of low income, rural residences, or limited English proficiency. From March 2020 to March 2022, this retrospective cohort study analyzed the use of in-person, phone/message, and telemedical care at a single tertiary care center in a oncology department. We investigated the association between economic, racial, ethnic, socioeconomic factors and forms of care, including in-person visit, telemedicine-based visit, and telephone/message. Study result shows that telemedicine utilization is lower among patients 65 and older, female patients, American Indian or Alaska Native patients, uninsured patients, and patients who require interpreters during clinical visits. As a result, it is unlikely that telemedicine will provide equal access to clinical care for all populations. On the other hand, In-person care utilization, remains low in low-income and rural-living patients compared to the general population, while telephone and message use remains high in low-income and rural-living patients. We conclude that telemedical care is currently unable to close the utilization gap for populations of low socioeconomic status. Patients with low socioeconomic status use in-person care less frequently. For the disadvantaged, unusually high telephone or message utilization is unlikely to provide the same quality as in-person or telemedical care. Understanding the causes of disparity and promoting a solution to improve equal access to care for all patients is critical.

   @article{tong2022telemedicine,
     title = {Telemedicine and Health Disparities: Association between Patient Characteristics and Telemedicine, In-person, Telephone and message-based Care During the COVID-19 Pandemic},
     author = {Tong, Ling and George, Ben and Crotty, Bradley and Melek, Somai and Taylor, Bradley and Osinski, Kristen and Luo, Jake},
     year = {2022},
     month = sep,
     journal = {IPEM Translation.},
   }

6. 

   Telemedicine adoption during the COVID-19 pandemic: gaps and inequalities

   Luo, Jake,  Tong, Ling,  Crotty, Bradley H and 4 more authors

   Applied clinical informatics. Aug 2021

   Abs Bib HTML PDF

   Background The telemedicine industry has been experiencing fast growth in recent years. The outbreak of coronavirus disease 2019 (COVID-19) further accelerated the deployment and utilization of telemedicine services. An analysis of the socioeconomic characteristics of telemedicine users to understand potential socioeconomic gaps and disparities is critical for improving the adoption of telemedicine services among patients. Objectives This study aims to measure the correlation of socioeconomic determinants with the use of telemedicine services in Milwaukee metropolitan area. Methods Electronic health record review of patients using telemedicine services compared with those not using telemedicine services within an academic-community health system: patient demographics (e.g., age, gender, race, and ethnicity), insurance status, and socioeconomic determinants obtained through block-level census data in Milwaukee area. The telemedicine users were compared with all other patients using regression analysis. The telemedicine adoption rates were calculated across regional ZIP codes to analyze the geographic patterns of telemedicine adoption. Results A total of 104,139 patients used telemedicine services during the study period. Patients who used video visits were younger (median age 48.12), more likely to be White (odds ratio [OR] 1.34; 95% confidence interval [CI], 1.31–1.37), and have private insurance (OR 1.43; CI, 1.41–1.46); patients who used telephone visits were older (median age 57.58), more likely to be Black (OR 1.31; CI 1.28–1.35), and have public insurance (OR 1.30; CI 1.27–1.32). In general, Latino and Asian populations were less likely to use telemedicine; women used more telemedicine services in general than men. In the multiple regression analysis of social determinant factors across 126 ZIP codes, college education (coefficient 1.41, p = 0.01) had a strong correlation to video telemedicine adoption rate. Conclusion Adoption of telemedicine services was significantly impacted by the social determinant factors of health, such as income, education level, race, and insurance type. The study reveals the potential inequities and disparities in telemedicine adoption.

   @article{luo2021telemedicine,
     title = {Telemedicine adoption during the COVID-19 pandemic: gaps and inequalities},
     author = {Luo, Jake and Tong, Ling and Crotty, Bradley H and Somai, Melek and Taylor, Bradley and Osinski, Kristen and George, Ben},
     journal = {Applied clinical informatics.},
     volume = {12},
     number = {04},
     pages = {836--844},
     year = {2021},
     month = aug,
     publisher = {Georg Thieme Verlag KG}
   }