Research | Ling Tong, PhD

2023

Understanding Population Gaps and Using Explainable Machine Learning to Predict Risk of Falls for Senior Adults

Tong, Ling

Doctorate Degree Thesis, University of Wisconsin-Milwaukee Sep 2023

HTML

Socioeconomic Disparities for Healthcare Utilization of Senior Adult Falls in Southeast Wisconsin, 2020-2022

Tong, Ling, Khani, Masoud, Taylor, Bradley and 4 more authors

medRxiv Apr 2023

@article{Tong2023socioeconomic,
  author = {Tong, Ling and Khani, Masoud and Taylor, Bradley and Osinski, Kristen and Adams, Jazzmyne and Friedland, David and Luo, Jake},
  title = {Socioeconomic Disparities for Healthcare Utilization of Senior Adult Falls in Southeast Wisconsin, 2020-2022},
  year = {2023},
  month = apr,
  doi = {10.1101/2023.04.24.23289062},
  publisher = {Cold Spring Harbor Laboratory Press},
  journal = {medRxiv},
}

Interpretable Machine Learning Text Classification for Clinical Computed Tomography reports – A Case Study of Temporal Bone Fracture

Tong, Ling, Luo, Jake, Adams, Jazzmyne and 3 more authors

Computer Methods and Programs in Biomedicine Update. Apr 2023

Abs Bib HTML

Background: Machine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model’s complexity, interpreting mechanisms of classification results remains difficult. Methods: We investigated interpretable representations of machine learning classification models. We created machine learning models to classify temporal bone fractures based on 164 temporal bone Computed Tomography (CT) text reports. We adopted the XGBoost, Support Vector Machine, Logistic Regression, and Random Forest algorithms. To interpret models, we used two major methodologies: (1) We calculated the average word frequency score (WFS) for keywords. The word frequency score shows the frequency gap between positive and negative classified cases. (2) We used Local Interpretable Model-Agnostic Explanations (LIME) to show the word-level contribution to bone fracture classification. Results: In temporal bone fracture classification, the random forest model achieved an average F1-score of 0.93. WFS reveals a difference in keyword usage between fracture and non-fracture cases. Additionally, LIME visualized the keywords’ contributions to classification results. The evaluation of LIME-based interpretation achieved the highest interpreting accuracy of 0.97. Conclusion: The interpretable text explainer can improve physicians’ understanding of machine learning predictions. By providing simple visualization, our model can increase the trust of computerized models. Our model supports more transparent computerized decision-making in clinical settings.
@article{tong2022interpretable, title = {Interpretable Machine Learning Text Classification for Clinical Computed Tomography reports – A Case Study of Temporal Bone Fracture}, author = {Tong, Ling and Luo, Jake and Adams, Jazzmyne and Liu, Xiaoyu and Osinski, Kathleen and Fridland, David}, year = {2023}, month = apr, journal = {Computer Methods and Programs in Biomedicine Update.}, }
MultiNet with Transformers: A Model for Cancer Diagnosis Using Images

Barzekar, H, Patel, Y, Tong, L and 1 more author

arXiv preprint arXiv:2301.09007 Jan 2023

Abs Bib HTML

Cancer is a leading cause of death in many countries. An early diagnosis of cancer based on biomedical imaging ensures effective treatment and a better prognosis. However, biomedical imaging presents challenges to both clinical institutions and researchers. Physiological anomalies are often characterized by slight abnormalities in individual cells or tissues, making them difficult to detect visually. Traditionally, anomalies are diagnosed by radiologists and pathologists with extensive training. This procedure, however, demands the participation of professionals and incurs a substantial cost. The cost makes large-scale biological image classification impractical. In this study, we provide unique deep neural network designs for multiclass classification of medical images, in particular cancer images. We incorporated transformers into a multiclass framework to take advantage of data-gathering capability and perform more accurate classifications. We evaluated models on publicly accessible datasets using various measures to ensure the reliability of the models. Extensive assessment metrics suggest this method can be used for a multitude of classification tasks.
@article{barzekar2023multinet, title = {MultiNet with Transformers: A Model for Cancer Diagnosis Using Images}, author = {Barzekar, H and Patel, Y and Tong, L and Yu, Z}, journal = {arXiv preprint arXiv:2301.09007}, year = {2023}, month = jan, }
The Demographics of Menière’s Disease: Selection Bias or Differential Susceptibility?

Thompson-Harvey, Adam, Friedland, David R, Adams, Jazzmyne A and 3 more authors

Otology & Neurotology Jan 2023

Abs Bib HTML PDF

Objective: To clarify whether the reported lack of racial and ethnic diversity among Menière’s disease (MD) patients is representative of selection bias or disease susceptibility. Study Design: Retrospective medical record review and populationlevel analyses. Setting: Tertiary referral center. Patients: Cohort of 1091 patients diagnosed with MD by the tertiary otology service. Main Outcome Measure: Demographic and population-level characteristics (age, sex, race, insurance status, ZIP code, median income, education level) compared with local, regional, health system, and otolaryngology clinic demographics. Results: Patients seen for MD were significantly older than those seen throughout the otolaryngology clinic (median, 65.0 versus 58.8 yr) or health system (65.0 versus 50.8 yr). A majority of patients with MD were of White race (92%), compared with 2.7% Black race and 0.5% Asian. Using population-level data, median income and having medical insurance were significantly correlated with care for MD. A disproportionate rate of care for MD was seen in ZIP codes outside urban areas as compared with other otologic and otolaryngologic conditions seen in the same clinic. Conclusion: Patients with MD are of older age, more likely to be of White race, and disproportionately from rural locales. The demographic profile of patients diagnosed with MD by tertiary otology is better explained by differential susceptibility to MD than by selection bias.
@article{thompson2022demographics, title = {The Demographics of Meni{\`e}re's Disease: Selection Bias or Differential Susceptibility?}, author = {Thompson-Harvey, Adam and Friedland, David R and Adams, Jazzmyne A and Tong, Ling and Osinski, Kristen and Luo, Jake}, journal = {Otology & Neurotology}, year = {2023}, month = jan, }

2022

Association between Body-Mass Index, Patient Characteristics, and Obesity-related Comorbidities among COVID-19 Patients: A Prospective Cohort Study

Tong, Ling, Khani, Masoud, Lu, Qiang and 3 more authors

Obesity Research and Clinical Practice Dec 2022

Abs Bib HTML

Objective: Obesity is a major risk factor for adverse outcomes after COVID-19 infection. However, it is unknown if the worse outcomes are due to the confounding effect of demographic and obesity-related comorbidities. The study objective is to analyze associations between body mass index, patient characteristics, obesity-related comorbidity, and clinical outcomes in COVID-19 patients. Methods: In this prospective cohort study, we chose patient records between March 1st, 2020, and December 1st, 2022, in a large tertiary care center in southeast Wisconsin in the United States. Patients over the age of 18 who tested positive were included in the study. Clinical outcomes included hospitalization, intensive care unit (ICU) admission, mechanical ventilation, and mortality rates. We examined the characteristics of patients who had positive clinical outcomes. We created unadjusted logistic regression models, sequentially adjusting for demographic and comorbidity variables, to assess the independent associations between BMI, patient characteristics, obesity-related comorbidities, and clinical outcomes. Results: From a record of 1.67 million inpatients and outpatients at Froedtert Health Center, 55,299 (BMI: 30.5±7.4 kg/m2, 62.5% female) tested COVID-19 positive during the study period. 17,580 (31.8%) patients were admitted to hospitals, and 2,856 (16.2%) of hospitalized patients required ICU admission. 1038 (36.3%) required mechanical ventilation, and 462 (44.5%) died after a positive test for COVID-19. We found female patients show a higher hospitalization rate, while male patients have a higher rate of ICU admission, mechanical ventilation, and mortality. Obesity-related comorbidities are associated with worse outcomes compared to simple obesity without comorbidities. In logistic regression models, we found four similar V-shaped associations between BMI and four clinical outcomes. Patients with a BMI of 25 kg/m2 are at the lowest risk for clinical outcomes. Patients with a BMI lower than 18 kg/m2 or higher than 30 kg/m2 are associated with a higher risk of hospitalization, ICU, mechanical ventilation, and death. After adjusting the model for demographic factors and hypertension and diabetes as two common comorbidities, we found that demographic factors do not significantly increase the risk. Obesity alone does not significantly increase the risk of severe clinical outcomes. Obesity-related comorbidities, on the other hand, resulted in a significantly higher risk of outcomes. Conclusion: Obesity alone does not increase the risk of worse clinical outcomes after COVID-19 infection. It may suggest that the worse clinical outcomes of patients with obesity are mediated via hypertension and type 2 diabetes. Patients with obesity and comorbidities have a higher risk of poor outcomes. Obesity-related comorbidities, including hypertension and diabetes, are independently associated with poorer clinical outcomes among COVID-19 patients. At a BMI of more than 30 kg/m2 or less than 18 kg/m2, we found an increase in the risk of severe COVID-19 outcomes leading to hospitalization, ICU, mechanical ventilation, and death. The increased risk of severe outcomes is not attributed to patient characteristics but can be attributed to hypertension and diabetes.
@article{tong2022association, title = {Association between Body-Mass Index, Patient Characteristics, and Obesity-related Comorbidities among COVID-19 Patients: A Prospective Cohort Study}, author = {Tong, Ling and Khani, Masoud and Lu, Qiang and Taylor, Bradley and Osinski, Kristen and Luo, Jake}, year = {2022}, month = dec, journal = {Obesity Research and Clinical Practice}, }
Visualizing the Interpretation of a Criteria-Driven System that Automatically Evaluates the Quality of Health News: An Exploratory Study of Two Approaches

Liu, Xiaoyu, Alsghaier, Hiba, Tong, Ling and 2 more authors

JMIR AI Nov 2022

Abs Bib HTML

Background: Machine learning techniques have been shown efficient at identifying health misinformation. However, interpreting a classification model remains challenging due to the model’s intricacy. The absence of a justification for the classification result and disclosure of the model’s domain knowledge may erode end-users’ trust in such models. This diminished trust may also undermine the effectiveness of artificial intelligence-based initiatives to counteract health misinformation. Objective: The study objective is to address the public’s need for help evaluating the quality of health news and the typical opaqueness of an AI approach. This work aims to create an interpretable, criteria-driven system. We also aim to find the best approach by comparing the accuracy of two methods for selecting and visualizing the system’s interpretation. Both methods highlight the sentences that contributed to the analysis performed by the system for a given criterion. Methods: We employ an interpretable, criteria-based approach for automatically assessing the quality of health news on the Internet. One of ten well-established criteria was chosen for our experimentation. To automate the evaluation of the criterion, we tested Logistic Regression, Naive Bayes, Support Vector Machine, and Random Forest algorithms. We then experimented with two approaches for developing interpretable representations of the results. For the first approach, (1) we calculate word feature weights, which explains how classification models distill keywords that are relevant to the positive prediction;(2) then using the Local Interpretable Model Explanations (LIME) framework, we automatically select keywords for visualization that show how classification models identify positive news articles, based on related keywords;(3)and finally, the system highlights target sentences containing keywords to justify the criterion evaluation result. For the second approach, (1) we extract sentences that provide evidence to support the evaluation result from 100 health news articles, (2) based on these results, we train a typology classification model at a sentence level; (3) then, the system highlights positive sentence instances for the result justification. We assess the accuracy of both methods using a small held-out test set. Results: The Random Forest model achieved the highest average AUC score of 0.74 in evaluating the criterion. Additionally, LIME provided a visualization of how keywords impact the system’s evaluation results. Both approaches could successfully visualize the interpretation of the system. When tested with four cases, the hybrid approach performs slightly better in highlighting correct sentences with an accuracy of 55.00%, compared to 50.00% by the typology approach. Conclusions: We provide an interpretable, criteria-based method to evaluate health news. The method incorporates rule-based and statistical machine learning approaches. Results suggest one might automate criterion-based health news quality evaluation successfully using either approach, but larger differences may arise when multiple quality-related criteria are considered. This work can increase public trust in computerized health information evaluation
@article{liu2022visualizing, title = {Visualizing the Interpretation of a Criteria-Driven System that Automatically Evaluates the Quality of Health News: An Exploratory Study of Two Approaches}, author = {Liu, Xiaoyu and Alsghaier, Hiba and Tong, Ling and Ataullah, Amna and Mcroy, Susan}, year = {2022}, month = nov, journal = {JMIR AI}, }
Telemedicine and Health Disparities: Association between Patient Characteristics and Telemedicine, In-person, Telephone and message-based Care During the COVID-19 Pandemic

Tong, Ling, George, Ben, Crotty, Bradley and 4 more authors

IPEM Translation. Sep 2022

Abs Bib HTML

Telemedicine has been an essential form of care since the onset of the COVID-19 pandemic. However, telemedicine may exacerbate disparities for populations with limited digital literacy or access, such as older adults, racial minorities, patients of low income, rural residences, or limited English proficiency. From March 2020 to March 2022, this retrospective cohort study analyzed the use of in-person, phone/message, and telemedical care at a single tertiary care center in a oncology department. We investigated the association between economic, racial, ethnic, socioeconomic factors and forms of care, including in-person visit, telemedicine-based visit, and telephone/message. Study result shows that telemedicine utilization is lower among patients 65 and older, female patients, American Indian or Alaska Native patients, uninsured patients, and patients who require interpreters during clinical visits. As a result, it is unlikely that telemedicine will provide equal access to clinical care for all populations. On the other hand, In-person care utilization, remains low in low-income and rural-living patients compared to the general population, while telephone and message use remains high in low-income and rural-living patients. We conclude that telemedical care is currently unable to close the utilization gap for populations of low socioeconomic status. Patients with low socioeconomic status use in-person care less frequently. For the disadvantaged, unusually high telephone or message utilization is unlikely to provide the same quality as in-person or telemedical care. Understanding the causes of disparity and promoting a solution to improve equal access to care for all patients is critical.
@article{tong2022telemedicine, title = {Telemedicine and Health Disparities: Association between Patient Characteristics and Telemedicine, In-person, Telephone and message-based Care During the COVID-19 Pandemic}, author = {Tong, Ling and George, Ben and Crotty, Bradley and Melek, Somai and Taylor, Bradley and Osinski, Kristen and Luo, Jake}, year = {2022}, month = sep, journal = {IPEM Translation.}, }
Standardization of Outcome Measures for Intratympanic Steroid Treatment for Idiopathic Sudden Sensorineural Hearing Loss

Osafo, Neil K, Friedland, David R, Harris, Michael S and 5 more authors

Otology & Neurotology. Oct 2022

Abs HTML PDF

Objective To identify variability in reported hearing outcomes for intratympanic (IT) steroid treatment of idiopathic sudden sensorineural hearing loss (ISSNHL) by comparing outcomes using the American Academy of Otolaryngology–Head and Neck Surgery (AAO-HNS) guideline with other published criteria. Study Design Retrospective chart review. Setting Tertiary otology practice. Patients Patients with ISSNHL treated with IT steroid between April 2003 and December 2020. Interventions IT steroid injection and audiometric evaluation. Main Outcome Measures 1: Rates of full, partial, or no recovery using the AAO-HNS guideline versus other reported criteria, and 2: correlation analyses of demographic and clinical variables with response to IT steroid. Results Using AAO-HNS reporting criteria, full recovery of the pure-tone average was noted in 25.68% of patients. Applying eight other published outcomes criteria to this patient cohort classified full recovery in 14.87 to 40.54% of patients. Similarly, AAO-HNS criteria classified “no recovery” in 51.35% of our patients, whereas applying the other reported criteria showed an average rate of 62.16% no recovery and as high as 82.43% of patients without recovery. Younger age (p = 0.003; effect size, 0.924) and IT injection within a week of onset (p < 0.001; effect size, 1.099) positively correlated with full recovery. There was no impact of prior or concurrent oral steroids, or number of steroid injections on outcome. Conclusion Great variability exists in the literature for assessment of IT steroid outcomes in ISSNHL. Standard reporting of outcomes as per the AAO-HNS SSNHL guideline is recommended to consistently characterize IT steroid efficacy and allow comparison across studies.
Accepted: Dr. Diagnosis: A Visualization Model for Diagnosing Diabetic Retinopathy Severity and Discovering Plaque Patterns in Retinal Images

Tong, Ling, Khani, Masoud, and Luo, Jake

International Conference on AI in Aging and Age-related Diseases. Oct 2022

Abs Bib

Diabetic Retinopathy(DR) is an age-related eye disease caused by the accumulation of plaque in the retina and often causes floaters, blurriness, dark areas of vision, and difficulty perceiving colors. The diagnosis of the DR requires a careful examination of retinal images, which makes it a time-intensive task for experienced clinicians. Currently, several deep neural networks have demonstrated higher diagnostic accuracy than clinical experts. However, it is difficult to interpret how artificial intelligence(AI) achieves transparent and interpretable diagnoses. We developed an AI-based tool, DR diagnosis, that visualizes the region-of-interest of retinal images for clinical providers to understand how AI diagnoses diabetic retinopathy. DR diagnosis uses retinal images as input, applies a Convolutional Neural Network to automatically find image patterns of plaque in the retina, and highlights the region-of-interest of the plaque in processed images. DR diagnosis can reduce the time-intensive inspection task, expedite the diagnosis process, and provide the transparency of AI-based tools. Physicians can be more likely to trust a transparent model and use our tool in clinical practices. We propose DR diagnosis has the potential to be integrated into clinical applications as a supportive visualization tool for future diabetic retinopathy diagnosis.
@article{tong2022dr, title = {Accepted: Dr. Diagnosis: A Visualization Model for Diagnosing Diabetic Retinopathy Severity and Discovering Plaque Patterns in Retinal Images}, author = {Tong, Ling and Khani, Masoud and Luo, Jake}, year = {2022}, month = oct, journal = {International Conference on AI in Aging and Age-related Diseases.}, }
Demographic differences in the treatment of unilateral vocal fold paralysis

Patel, Mit A., Bock, Jonathan M., Blumin, Joel H. and 5 more authors

Laryngoscope Investigative Otolaryngology. Aug 2022

Abs Bib HTML PDF

Objective To determine the impact of patient demographics and social determinants of health on treatment pathways for unilateral vocal fold paralysis (UVFP) at a tertiary laryngology clinic. Study design Retrospective medical record review. Methods Patient demographics (age, gender, race, ethnicity, and insurance status) were extracted for adults diagnosed with UVFP between 2009 and 2019. Odds ratios for the associations between sociodemographic factors and UVFP treatment pathways were determined by chi-square analyses. Results A total of 1490 UVFP diagnoses were identified during the study period with the majority being female (58%), White (85%), non-Hispanic (97%), and publicly insured (54%). Five treatment pathways were identified: observation, injection laryngoplasty, voice therapy, laryngeal framework surgery/thyroplasty, and reinnervation surgery. There were 538 patients who underwent observation, 512 injection laryngoplasty, 366 voice therapy, 136 thyroplasty, and 26 laryngeal reinnervation surgery. Males were more likely to undergo injection laryngoplasty than females (OR 1.32; CI 1.08–1.61), whereas females were more likely to undergo voice therapy (OR 1.39; CI 1.09–1.76). Patients with public insurance (OR 1.48; CI 1.03–2.14) and Hispanics (OR 2.60; CI 1.18–5.72) were more likely to undergo thyroplasty. Patients who underwent reinnervation surgery were younger than those in other treatment pathways (median: 39.1 years vs. 50.7–56.1 years). Conclusions Gender, ethnicity, and insurance status were significantly associated with specific UVFP treatment pathways. Patients with public insurance were more likely to undergo surgical intervention than voice therapy. This data overall supports differences in care pathway utilization for UVFP based on social determinants of health. Level of evidence Level IV.
@article{mit2022demographic, author = {Patel, Mit A. and Bock, Jonathan M. and Blumin, Joel H. and Friedland, David R. and Adams, Jazzmyne A. and Tong, Ling and Osinski, Kristen I. and Luo, Jake}, title = {Demographic differences in the treatment of unilateral vocal fold paralysis}, journal = {Laryngoscope Investigative Otolaryngology.}, volume = {n/a}, year = {2022}, month = aug, number = {n/a}, pages = {}, keywords = {demographics, health care disparities, laryngology, treatment, unilateral vocal fold paralysis}, doi = {https://doi.org/10.1002/lio2.920}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/lio2.920}, }
Impact of Demographics and Clinical Features on Initial Treatment Pathway for Vestibular Schwannoma

Harvey, Erin, Stark, Katarina, Friedland, David R and 5 more authors

Otology & Neurotology. Sep 2022

Abs Bib HTML PDF

Objective To identify demographic and clinical features impacting initial treatment pathway for vestibular schwannoma. Study Design Retrospective chart review. Setting Tertiary care academic medical center. Patients Patients diagnosed with vestibular schwannoma between 2009 and 2019. Interventions Observation, stereotactic radiosurgery, or microsurgical resection. Main Outcome Measures χ2 Test, one-way analysis of variance, and multivariate logistic regression were used to correlate demographic and clinical factors with initial treatment pathway for 197 newly diagnosed vestibular schwannoma patients. Results Among 197 patients, 93 (47%) were initially treated with observation, 60 (30%) with stereotactic radiation (Gamma Knife) and 44 (22%) with surgical resection. Age univariately had no statistically significant impact on initial pathway, but those undergoing surgery trended toward a younger demographic (49.1 yr [surgery] versus 57.2 yr [observation] versus 59.0 yr [Gamma Knife]). Men were more likely to be initially observed than women (p = 0.04). Patients initially observed were more likely to have a lower Koos classification (p < 0.001) and have better tumor-ear hearing (p = 0.03). Only 34.4% of patients living outside the local geographic region were initially observed compared with 53.0% living locally (p = 0.055). Surgeon correlated with initial treatment (p = 0.04) but did not maintain significance when adjusting for hearing level or tumor size. A multiple linear regression model found age, maximum tumor diameter, and Koos class to correlate with initial treatment pathway (p < 0.0001, r2 = 0.42). Conclusion Initial treatment pathway for newly diagnosed vestibular schwannoma is impacted by demographic factors such as age, sex, and geographic proximity to the medical center. Clinical features including hearing level and tumor size also correlated with initial treatment modality.
@article{harvey2022impact, title = {Impact of Demographics and Clinical Features on Initial Treatment Pathway for Vestibular Schwannoma}, author = {Harvey, Erin and Stark, Katarina and Friedland, David R and Adams, Jazzmyne A and Harris, Michael S and Tong, Ling and Osinksi, Kristen and Luo, Jake}, journal = {Otology \& Neurotology.}, volume = {43}, number = {9}, pages = {1078--1084}, year = {2022}, month = sep, publisher = {LWW} }
Analysis of socioeconomic factors in laryngology clinic utilization for treatment of dysphonia

White, Shane W, Bock, Jonathan M, Blumin, Joel H and 5 more authors

Laryngoscope Investigative Otolaryngology. Aug 2022

Abs Bib HTML PDF

Objective To evaluate the association between patient socioeconomic and demographic factors and tertiary care utilization for dysphonia in a localized metropolitan area of the American Midwest. Methods Multivariate regression analysis was used to correlate patient demographics and population level data (e.g., age, gender, race, insurance, median income, education level) with tertiary laryngology utilization for dysphonia care at our institution between 2000 and 2019. Initial analyses characterized tertiary laryngology utilization rates for all regional ZIP codes and correlated these data with census information for household income and education. Dysphonia patient demographics were compared among populations cared for in our entire academic Otolaryngology department, our health system, and the regional population. Results Among 1,365,021 patients in our health system, there were 7066 tertiary laryngology visits with a diagnosis of dysphonia. Dysphonia patients as compared to the overall health system were older (62.0 vs. 50.8 years), more likely to be female (63.7 vs. 50.2%) and more likely to have insurance (98.4 vs. 87.5%, all p < .001). Patient and population-level factors including insurance status, education, and black race showed positive correlation with laryngology utilization while median income did not. Conclusions and Relevance Insurance status, education level, and race correlated with utilization of tertiary laryngology services for the evaluation of dysphonia in our community, while median income did not. Black patients utilized tertiary laryngology care at higher rates compared to departmental and regional population utilization data. These results underscore important demographic and disease-specific factors that may affect utilization of subspecialty care in Otolaryngology.
@article{white2022analysis, title = {Analysis of socioeconomic factors in laryngology clinic utilization for treatment of dysphonia}, author = {White, Shane W and Bock, Jonathan M and Blumin, Joel H and Friedland, David R and Adams, Jazzmyne A and Tong, Ling and Osinski, Kristen and Luo, Jake}, journal = {Laryngoscope Investigative Otolaryngology.}, volume = {7}, number = {1}, pages = {202--209}, year = {2022}, month = aug, publisher = {Wiley Online Library} }
A Clustering-Aided Approach for Diagnosis Prediction: A Case Study of Elderly Fall

Tong, Ling, Luo, Jake, Adams, Jazzmyne and 3 more authors

In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC). Jun 2022

Abs Bib HTML PDF

Data-driven diagnosis prediction has been adopted in clinical decision support systems. However, only a few studies have focused on non-supervised clustering approaches to building a high-quality patient data set. This study focused on a clustering-aided approach to diagnosis prediction. We leveraged clustering-aided machine learning models to predict elderly falls. First, we used patients’ risk factors to build a feature set. The feature set showed a clustering-aided approach could aggregate patient factors that shared similar clinical and demographic characteristics. Subsequently, a K-means clustering approach significantly improved the data set quality. Overall, our study demonstrated that clustering approaches improve the prediction performance of elderly falls. A clustering-aided approach can be applied to similar clinical healthcare practices to potentially improve elderly care.
@inproceedings{tong2022clustering, title = {A Clustering-Aided Approach for Diagnosis Prediction: A Case Study of Elderly Fall}, author = {Tong, Ling and Luo, Jake and Adams, Jazzmyne and Osinski, Kristen and Liu, Xiaoyu and Friedland, David}, booktitle = {2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC).}, pages = {337--342}, year = {2022}, month = jun, organization = {IEEE} }
The impact of social determinants of health and clinical comorbidities on post-tympanotomy tube otorrhea

Thomas, Abigail, Flanary, Valerie, Friedland, David R and 4 more authors

International Journal of Pediatric Otorhinolaryngology. Jan 2022

Abs Bib HTML PDF

Objectives To measure the impact of social determinants of health and clinical comorbidities on the incidence of post-tympanotomy tube otorrhea (PTTO). Methods Retrospective observational cohort study. All children between the ages of 0 and 17 having tympanotomy tube placement between 2009 and 2019. Between group comparisons entailed the calculation of odds ratios (OR) with 95% confidence intervals and associated p-values. Results Among 12,757 patients who underwent myringotomy and tube placement, 2217 (17.4%) presented with PTTO within 1 year. Race and sex did not correlate with the development of PTTO. Non-Hispanic ethnicity had a negative association with PTTO (OR: 0.80 (0.70-0.91), p < .0001). Insurance status correlated with incidence of PTTO with a higher rate noted among those with public insurance (OR: 1.12 (1.02–1.23), p = .02) and a lower rate among those with private insurance (OR: 0.84 (0.77-0.92), p < .0001). Craniofacial abnormalities had the strongest positive correlation with PTTO, particularly, cleft lip and/or cleft palate (OR>2.24, p < .0001). Immunodeficiency had similar impact on PTTO (OR: 2.38 (1.46–3.91), p < .0001). Asthma and prematurity did not significantly correlate with occurrence of PTTO. Conclusion Higher rates of PTTO correlated strongest with clinical factors; particularly craniofacial abnormalities and immunodeficiency. Social determinants, including private insurance and non-Hispanic ethnicity, were associated with lower rates of PTTO. Race and sex did not show significant correlations.
@article{thomas2022impact, title = {The impact of social determinants of health and clinical comorbidities on post-tympanotomy tube otorrhea}, author = {Thomas, Abigail and Flanary, Valerie and Friedland, David R and Adams, Jazzmyne A and Tong, Ling and Osinski, Kristen and Luo, Jake}, journal = {International Journal of Pediatric Otorhinolaryngology.}, volume = {152}, pages = {110986}, year = {2022}, month = jan, publisher = {Elsevier} }

2021

Telemedicine adoption during the COVID-19 pandemic: gaps and inequalities

Luo, Jake, Tong, Ling, Crotty, Bradley H and 4 more authors

Applied clinical informatics. Aug 2021

Abs Bib HTML PDF

Background The telemedicine industry has been experiencing fast growth in recent years. The outbreak of coronavirus disease 2019 (COVID-19) further accelerated the deployment and utilization of telemedicine services. An analysis of the socioeconomic characteristics of telemedicine users to understand potential socioeconomic gaps and disparities is critical for improving the adoption of telemedicine services among patients. Objectives This study aims to measure the correlation of socioeconomic determinants with the use of telemedicine services in Milwaukee metropolitan area. Methods Electronic health record review of patients using telemedicine services compared with those not using telemedicine services within an academic-community health system: patient demographics (e.g., age, gender, race, and ethnicity), insurance status, and socioeconomic determinants obtained through block-level census data in Milwaukee area. The telemedicine users were compared with all other patients using regression analysis. The telemedicine adoption rates were calculated across regional ZIP codes to analyze the geographic patterns of telemedicine adoption. Results A total of 104,139 patients used telemedicine services during the study period. Patients who used video visits were younger (median age 48.12), more likely to be White (odds ratio [OR] 1.34; 95% confidence interval [CI], 1.31–1.37), and have private insurance (OR 1.43; CI, 1.41–1.46); patients who used telephone visits were older (median age 57.58), more likely to be Black (OR 1.31; CI 1.28–1.35), and have public insurance (OR 1.30; CI 1.27–1.32). In general, Latino and Asian populations were less likely to use telemedicine; women used more telemedicine services in general than men. In the multiple regression analysis of social determinant factors across 126 ZIP codes, college education (coefficient 1.41, p = 0.01) had a strong correlation to video telemedicine adoption rate. Conclusion Adoption of telemedicine services was significantly impacted by the social determinant factors of health, such as income, education level, race, and insurance type. The study reveals the potential inequities and disparities in telemedicine adoption.
@article{luo2021telemedicine, title = {Telemedicine adoption during the COVID-19 pandemic: gaps and inequalities}, author = {Luo, Jake and Tong, Ling and Crotty, Bradley H and Somai, Melek and Taylor, Bradley and Osinski, Kristen and George, Ben}, journal = {Applied clinical informatics.}, volume = {12}, number = {04}, pages = {836--844}, year = {2021}, month = aug, publisher = {Georg Thieme Verlag KG} }
A deep learning study on osteosarcoma detection from histological images

Anisuzzaman, DM, Barzekar, Hosein, Tong, Ling and 2 more authors

Biomedical Signal Processing and Control. May 2021

Abs Bib HTML PDF

In the U.S. 5–10% of new pediatric cases of cancer are primary bone tumors. The most common type of primary malignant bone tumor is osteosarcoma. The intention of the present work is to improve the detection and diagnosis of osteosarcoma using computer-aided detection (CAD) and diagnosis (CADx). Such tools as convolutional neural networks (CNNs) can significantly decrease the surgeon’s workload and make a better prognosis of patient conditions. CNNs need to be trained on a large amount of data in order to achieve a more trustworthy performance. In this study, transfer learning techniques, pre-trained CNNs, are adapted to a public dataset on osteosarcoma histological images to detect necrotic images from non-necrotic and healthy tissues. First, the dataset was preprocessed, and different classifications are applied. Then, Transfer learning models including VGG19 and Inception V3 are used and trained on Whole Slide Images (WSI) with no patches, to improve the accuracy of the outputs. Finally, the models are applied to different classification problems, including binary and multi-class classifiers. Experimental results show that the accuracy of the VGG19 has the highest, 96%, performance amongst all binary classes and multiclass classification. Our fine-tuned model demonstrates state-of-the-art performance on detecting malignancy of Osteosarcoma based on histologic images.
@article{anisuzzaman2021deep, title = {A deep learning study on osteosarcoma detection from histological images}, author = {Anisuzzaman, DM and Barzekar, Hosein and Tong, Ling and Luo, Jake and Yu, Zeyun}, journal = {Biomedical Signal Processing and Control.}, volume = {69}, pages = {102931}, year = {2021}, month = may, publisher = {Elsevier} }
Socioeconomic Determinants of Tertiary Rhinology Care Utilization

Poetker, David M, Friedland, David R, Adams, Jazzmyne A and 3 more authors

OTO open. Mar 2021

Abs Bib HTML PDF

Objective The objective of this study was to determine the impact of patient demographics and socioeconomic factors on the utilization of tertiary rhinology care services in an upper Midwestern academic medical center. Study Design Retrospective review of electronic health records. Setting Academic medical center. Methods The electronic health record of our academic center was interrogated for the demographics and diagnosis of chronic rhinosinusitis (CRS) among adult patients seen by fellowship-trained rhinologists from 2000 to 2019. Patient characteristics (age, sex, race, insurance status) and population-level data (median income and education level) were compared with utilization of tertiary rhinology services for CRS. Utilization rates were calculated for each regional zip code and correlated with census data for median income and education. The association between determinants of health and tertiary rhinology utilization was assessed by multivariate regression analyses. Results A total of 8325 patients diagnosed with CRS used tertiary rhinology services. Patients were older (median, 58.9 years) and more likely to be female (57.6%), White (85%), and privately insured (60%) when compared with patients seen across our hospital system (P < .001). Adjusted analyses showed median income, education level, and White race to be independently correlated with tertiary care utilization. Private insurance alone was not an independent contributing factor to access. Conclusion Utilization of tertiary rhinology services correlated with income, race, and education level. Private insurance was not an independent factor. These results highlight social differences in determinants of access to tertiary otolaryngologic care.
@article{poetker2021socioeconomic, title = {Socioeconomic Determinants of Tertiary Rhinology Care Utilization}, author = {Poetker, David M and Friedland, David R and Adams, Jazzmyne A and Tong, Ling and Osinski, Kristen and Luo, Jake}, journal = {OTO open.}, volume = {5}, number = {2}, pages = {2473974X211009830}, year = {2021}, month = mar, publisher = {SAGE Publications Sage CA: Los Angeles, CA} }

2020

tu1058 Association between attending endoscopists’experience and complication rates for all endoscopic procedures: a 10-year longitudinal study

Hernandez, Lyndon, Tong, Ling, Cofino, Julia and 4 more authors

Gastrointestinal Endoscopy. May 2020

Abs Bib HTML PDF

Objective The objective of this study was to determine the impact of patient demographics and socioeconomic factors on the utilization of tertiary rhinology care services in an upper Midwestern academic medical center. Study Design Retrospective review of electronic health records. Setting Academic medical center. Methods The electronic health record of our academic center was interrogated for the demographics and diagnosis of chronic rhinosinusitis (CRS) among adult patients seen by fellowship-trained rhinologists from 2000 to 2019. Patient characteristics (age, sex, race, insurance status) and population-level data (median income and education level) were compared with utilization of tertiary rhinology services for CRS. Utilization rates were calculated for each regional zip code and correlated with census data for median income and education. The association between determinants of health and tertiary rhinology utilization was assessed by multivariate regression analyses. Results A total of 8325 patients diagnosed with CRS used tertiary rhinology services. Patients were older (median, 58.9 years) and more likely to be female (57.6%), White (85%), and privately insured (60%) when compared with patients seen across our hospital system (P < .001). Adjusted analyses showed median income, education level, and White race to be independently correlated with tertiary care utilization. Private insurance alone was not an independent contributing factor to access. Conclusion Utilization of tertiary rhinology services correlated with income, race, and education level. Private insurance was not an independent factor. These results highlight social differences in determinants of access to tertiary otolaryngologic care.
@article{hernandez2020tu1058, title = {tu1058 Association between attending endoscopists'experience and complication rates for all endoscopic procedures: a 10-year longitudinal study}, author = {Hernandez, Lyndon and Tong, Ling and Cofino, Julia and Johannessen, Jack O and Guda, Nalini M and Muddana, Venkata and Luo, Jake}, journal = {Gastrointestinal Endoscopy.}, volume = {91}, number = {6}, pages = {AB525}, year = {2020}, month = may, publisher = {Elsevier} }
41 PREDICTING GASTROINTESTINAL (GI) HEMORRHAGE USING A MACHINE LEARNING APPROACH: RISK FACTORS AND PREDICTIVE ANALYSIS IN CLINICAL STUDIES

Tong, Ling, Hernandez, Lyndon V, and Luo, Jake

Gastroenterology. May 2020

Abs Bib HTML PDF

Background: Hospital systems and third party payers are interested in the frequency of adverse events as a quality indicator. Prior studies have demonstrated patient and procedure-related factors associated with complication rates for all endoscopic procedures, but there is limited information on the effect of endoscopist’s experience on complication rates. Aim: To determine the association between attending endoscopists’ experience and complication rates for all endoscopic procedures. Methods This is a retrospective longitudinal study analyzing the complication database of a large practice group from January 1, 2007 to December 2016. Major complications (unexpected admissions or surgery, prolonged hospitalizations, deaths) are self-reported into our database that is managed by a nurse. Procedures were performed in hospitals (tertiary and community) and ambulatory surgery centers. Endoscopist experience is defined as the period from year of completing gastroenterology fellowship to when the complication occurred. Results: There were 163,591 endoscopic procedures performed by 13 endoscopists during the study period. The median age of patients was 58 years. The endoscopic procedures were colonoscopy (n=111), esophagogastroduodenoscopy (EGD) (n=50), and endoscopic retrograde cholangiopancreatography (ERCP) (n=66). There were a total of 213 complications (perforations, n=71; bleeding, n= 75; post-ERCP pancreatitis, n= 42). The median experience of endoscopists at the time of complications was 14 years (range: 1 - 24 years). When comparing endoscopists with less experience (1-10 years) with more experienced endoscopists (>10 years), a t-test shows that the more experienced endoscopists had a lower median complication rate (0.13% vs 0.27%, p=0.071). Using linear regression modeling, there was significant inverse relationship between endoscopists’ experience and complication rates (Figure 1), with a mean decrease of 0.091 percent per 1000 procedures performed (p=0.015). Limitations: Potential under-reporting of complications from self-reporting; no risk-adjustment in our preliminary analysis; some ERCPs performed with advanced endoscopy fellows. Conclusions: In our preliminary analysis of a large database, more experienced physicians in general have a lower risk for complications over a span of 25 years, although there was variation in performance. Future studies should focus on determining the likelihood that variation in these rare events will be detected.
@article{tong202041, title = {41 PREDICTING GASTROINTESTINAL (GI) HEMORRHAGE USING A MACHINE LEARNING APPROACH: RISK FACTORS AND PREDICTIVE ANALYSIS IN CLINICAL STUDIES}, author = {Tong, Ling and Hernandez, Lyndon V and Luo, Jake}, journal = {Gastroenterology.}, volume = {158}, number = {6}, pages = {S--16}, year = {2020}, month = may, publisher = {Elsevier} }
Tu1981: Association modeling between patients’ age and complication rate for endoscopic procedures

Tong, Ling, Hernandez, Lyndon V, Cofino, Julia and 3 more authors

Gastroenterology. Jun 2020

Abs Bib HTML PDF

Background: The likelihood of observing complications after an endoscopic procedure is believed to relate to patient age. Prior studies have demonstrated that older patients (age>70) are more likely to experience complications than younger patients. However, currently there is still a lack of knowledge of the quantified association between patient age and complication across all age groups. Aim: This study is a preliminary attempt to model the quantitative association between patients’ ages and complication rates for all endoscopic procedures. Methods: Leveraging the complication database of a large practice group from January 1, 2007 to December 2016, we conducted a retrospective longitudinal study to model the complication rates across different age groups. The major complications (unexpected admissions or surgery, prolonged hospitalizations, deaths) were self-reported into the complication database that was managed by nurses. Patient age was calculated when the patient was admitted into the clinics. Procedures were performed in hospital and ambulatory surgery center settings. Results: We collected the data of 163,591 endoscopic procedures performed between January 1, 2007 and December 31, 2016. There was a total of 213 documented complications. The top 3 complications include 75 cases of bleeding, 71 cases of perforations, and 42 cases of post-ERCP pancreatitis. The endoscopic procedures associated with these complications include 111 cases of colonoscopy, 66 cases of endoscopic retrograde cholangiopancreatography (ERCP), and 50 cases of esophagogastroduodenoscopy (EGD). ( Figure 1) Comparing three broad age groups (Young: age<=20, Medium: 21<=age<= 59, and Elderly: age>=60), the young age group (age<=20) had the highest average complication rate of 0.43%, followed by elderly patent group (age>=60) with a mean rate of 0.182%. The medium age group (21<=age<= 59) had the lowest complication rate of 0.126%. The differences in complication rates between different age groups are significant (p < 0.01 for any group pairs). Using regression modelling, a statistically significant (p < 0.001) quadratic relationship can be established between the patients’ ages and complication rates for all endoscopic procedures (Figure 2). On average, the complication rate is minimal among 52.1 years old patients. Conclusion: There is a significant difference of complication rates of endoscopic procedures across different age groups ( Figure 1). Our regression modeling study shows that there is a strong quadratic relationship between ages and complications ( Figure 2). Patient age could be a core factor for estimating the complication risks of endoscopic procedures. Our future study will focus on 1) improving the model by incorporating more data and risk adjustments, and 2) creating an evidence-based protocol for evaluating the likelihood of endoscopic procedure complications.
@article{tong2020tu1981, title = {Tu1981: Association modeling between patients' age and complication rate for endoscopic procedures}, author = {Tong, Ling and Hernandez, Lyndon V and Cofino, Julia and Johannessen, Jack O and Guda, Nalini M and Luo, Jake}, year = {2020}, month = jun, journal = {Gastroenterology.}, }

2019

Machine learning-based modeling of big clinical trials data for adverse outcome prediction: A case study of death events

Tong, Ling, Luo, Jake, Cisler, Ron and 1 more author

In 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC). Jun 2019

Abs Bib HTML PDF Poster

It is known that clinical trials have potential risks for participants, which could result in unexpected adverse events. To quantify and predict the risk of adverse outcomes, we leverage a large amount of clinical reports to build machine learning models to predict adverse outcomes. We focused on death events as the predicting target in this study. From Clinicaltrial.gov, we collected 28,340 reports and transformed the data into vectorized machine learning features. These features were harmonized across studies using semantic mapping and feature selection techniques. The resulting selected clinical trial features were used to build five machine learning models for prediction. We evaluated and compared relative model performances for the prediction task. Results show that the logistic regression algorithm achieved the best overall receiver operating characteristic score at 0.7344. This exploratory study showed that it is feasible to use clinical trial factors to predict adverse outcomes. We demonstrated the approach by focusing on building machine learning models to predict death outcomes. Predicting adverse outcomes could help clinical trials estimate harmful risks and design better mechanisms to protect participants. We hope by using our models, a clinical trial expert will be able to assess whether serious adverse events are likely to occur in a clinical trial at the early stage and to estimate what potential trial factors could contribute to the potential serious adverse events.
@inproceedings{tong2019machine, title = {Machine learning-based modeling of big clinical trials data for adverse outcome prediction: A case study of death events}, author = {Tong, Ling and Luo, Jake and Cisler, Ron and Cantor, Michael}, booktitle = {2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC).}, volume = {2}, pages = {269--274}, year = {2019}, month = jun, organization = {IEEE}, }

2017

Evaluating the granularity balance of hierarchical relationships within large biomedical terminologies towards quality improvement

Luo, Lingyun, Tong, Ling, Zhou, Xiaoxi and 3 more authors

Journal of Biomedical Informatics. Oct 2017

Abs Bib HTML PDF

Organizing the descendants of a concept under a particular semantic relationship may be rather arbitrarily carried out during the manual creation processes of large biomedical terminologies, resulting in imbalances in relationship granularity. This work aims to propose scalable models towards systematically evaluating the granularity balance of semantic relationships. We first utilize “parallel concepts set (PCS)” and two features (the length and the strength) of the paths between PCSs to design the general evaluation models, based on which we propose eight concrete evaluation models generated by two specific types of PCSs: single concept set and symmetric concepts set. We then apply those concrete models to the IS-A relationship in FMA and SNOMED CT’s Body Structure subset, as well as to the Part-Of relationship in FMA. Moreover, without loss of generality, we conduct two additional rounds of applications on the Part-Of relationship after removing length redundancies and strength redundancies sequentially. At last, we perform automatic evaluation on the imbalances detected after the final round for identifying missing concepts, misaligned relations and inconsistencies. For the IS-A relationship, 34 missing concepts, 80 misalignments and 18 redundancies in FMA as well as 28 missing concepts, 114 misalignments and 1 redundancy in SNOMED CT were uncovered. In addition, 6,801 instances of imbalances for the Part-Of relationship in FMA were also identified, including 3,246 redundancies. After removing those redundancies from FMA, the total number of Part-Of imbalances was dramatically reduced to 327, including 51 missing concepts, 294 misaligned relations, and 36 inconsistencies. Manual curation performed by the FMA project leader confirmed the effectiveness of our method in identifying curation errors. In conclusion, the granularity balance of hierarchical semantic relationship is a valuable property to check for ontology quality assurance, and the scalable evaluation models proposed in this study are effective in fulfilling this task, especially in auditing relationships with sub-hierarchies, such as the seldom evaluated Part-Of relationship.
@article{luo2017evaluating, title = {Evaluating the granularity balance of hierarchical relationships within large biomedical terminologies towards quality improvement}, author = {Luo, Lingyun and Tong, Ling and Zhou, Xiaoxi and Mejino Jr, Jose LV and Ouyang, Chunping and Liu, Yongbin}, journal = {Journal of Biomedical Informatics.}, volume = {75}, pages = {129--137}, year = {2017}, month = oct, publisher = {Elsevier} }
用微博预测疾病——基于社交网络数据的流感追踪模型

Tong, Ling

Bachelor’s Degree Thesis, University of South China Jun 2017

Abs PDF

社交网络产生的数据量不断增大，使得利用其隐含信息进行数据分析和挖掘成为可能。流行性感冒在地区性传播的速度很快，且具有高峰爆发的趋势，且很难对流感爆发的时间和地点进行预测，一旦发病，对公共卫生事业造成的损失巨大。在地区性传染病的监控中，主要依赖于医院的就诊数据。目前只有一部分患者去医院就诊，因此从医院得到的就诊数据具有时效性不强，完整度不高的特点。为了提高监测的及时性与准确性，在社交网络的信息中，根据相关数据以及地理位置信息，对特定病症进行文本分类，构建社交网络数据集。统计社交网络数据集疑似流感病例的数量，统计社交网络信息与实际发病病例的相关度。在保证数据的时效性和准确性的基础上，利用对未来可能发生爆发的地区、爆发规模和爆发时间进行预测。该预测模型使用新浪微博数据作为数据源，针对流行性感冒进行监控和预测。结合中国疾病预防控制中心给出的传染病的定义，发病模式等信息，归纳出其影响因子结合使用关键字过滤算法以及相关文本分类算法，从海量数据中提取出有效病例，并使用中国疾病预防控制中心的官方数据进行趋势验证。将经过分类算法分类后的数据作为基础，考虑用户社交关系以及地理位置对流感传播的影响，根据历史数据对模型的追踪效果进行评估。