Multi-Platform Sentiment Analysis of Diabetes Mellitus on X and TikTok Using K-Nearest Neighbor, Chi-Square Selection, and Oversampling

Abstract

Diabetes mellitus is a chronic health condition that is widely discussed by the public on social media, generating a large volume of opinions that are difficult to interpret manually. This study analyzes public sentiment toward Diabetes mellitus using data collected from X (Twitter) and TikTok. Text data were preprocessed (cleaning, slang normalization, stopword removal, and stemming) and duplicate entries were removed. Sentiment labels were generated automatically through a rule-based lexicon across four categories (positive, negative, neutral, and irrelevant/discarded), and the reliability of this automatic labeling was verified through manual validation of a stratified sample of 200 data points, measured using Cohen's Kappa. Positive and negative data were then weighted using TF-IDF, reduced using Chi-Square feature selection, balanced using SMOTE, and classified using K-Nearest Neighbor (KNN). Model performance was evaluated using accuracy, precision, recall, and F1-score, and compared across four scenarios: baseline KNN, KNN with Chi-Square, KNN with SMOTE, and the combined KNN+Chi-Square+SMOTE model. The manual validation of 198 valid samples produced an agreement accuracy of 59.60% and a Cohen's Kappa of 0.459 (moderate agreement), indicating that the main source of disagreement lies at the boundary between the neutral and sentiment-bearing classes, while direct positive-negative misclassification was rare (4.5%). The combined model achieved an accuracy of 82.86%, with a macro-averaged precision of 82.95%, recall of 83.56%, and F1-score of 82.79%. Interestingly, Chi-Square feature selection alone yielded the highest accuracy among the four scenarios (85.10%), suggesting that feature selection contributed more to performance gains than class balancing in this dataset. These findings suggest that combining feature selection and oversampling techniques improves the reliability of multi-platform sentiment classification for health-related topics and can inform more effective public health communication strategies regarding diabetes.